Mailman 3 April 2016 - Python-Dev

tp_new selection regression in the 2.7 branch
by Julien Cristau April 11, 2016

April 11, 2016

Hi, changeset https://hg.python.org/cpython/rev/e7062dd9085e in the 2.7 branch changes how tp_new is assigned, and causes regressions with multiple inheritance from extension classes. http://bugs.python.org/issue25731#msg262922 has a fairly simple reproducer using cython. The __base__ attribute is set correctly, but tp_new is now wrong and thus the object initialization is broken. Can this change be fixed or reverted before the next 2.7.x release? (I have not verified if this regression also affects the 3.5 branch) Thanks, Julien

1 0

Re: [Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)
by Wes Turner April 10, 2016

April 10, 2016

On Apr 10, 2016 11:51 AM, "Jon Ribbens" <jon+python-dev(a)unequivocal.co.uk> wrote: > > On Sun, Apr 10, 2016 at 02:51:23PM +1000, Nick Coghlan wrote: > > On 9 April 2016 at 22:43, Victor Stinner <victor.stinner(a)gmail.com> wrote: > > > See pysandbox test suite for a lot of ways to escape a sandbox. CPython has > > > a list of know code to crash CPython (I don't recall the dieectory in > > > sources), even with the latest version of CPython. > > > > They're at https://hg.python.org/cpython/file/tip/Lib/test/crashers > > Thanks. I take your point that sandboxing Python requires CPython to > free of code execution bugs. However I will note that none of the > crashers in that directory will work inside my experiment (except > "infinite_loop_re.py", which isn't a crasher just a long loop). > > > Even without those considerations though, there are system level > > denial of service attacks that untrusted code can perform without even > > trying to break out of the sandbox - the most naive is "while 1: > > pass", but there are more interesting ones like "from itertools import > > count; sum(count())", or even "sum(iter(int, 1))" and "list(iter(int, > > 1))". > > Yes, of course. I have already explicitly noted that infinite loops > and memory exhausation are not preventable. > > > Operating system level security sandboxes still aren't particularly > > easy to use correctly, but they're a lot more reliable than language > > runtime level sandboxes, can be used to defend against many more > > attack vectors, and even offer increased flexibility (e.g. "can write > > to these directories, but no others", "can read these files, but no > > others", "can contact these IP addresses, but no others"). > > I don't entirely trust operating system sandboxes either - I generally > assume that if someone can execute arbitrary code on my machine, then > they can do anything they want to that machine. > > What I *might* trust, though, would be a "sandbox Python" that is > itself running inside an operating system sandbox... > * https://github.com/jupyter/jupyterhub/wiki/Spawners - Docker LXC Containers - https://github.com/jupyter/jupyterhub/wiki/Authenticators - DOS is still trivial - Segfault is still trivial * http://doc.pypy.org/en/latest/sandbox.html#introduction _______________________________________________ > Python-Dev mailing list > Python-Dev(a)python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com

1 0

When should pathlib stop being provisional?
by Brett Cannon April 10, 2016

April 10, 2016

After a rather extensive discussion on python-ideas about pathlib.PurePath not inheriting from str, another point that came up was that the use of pathlib has been rather light. Unfortunately even the stdlib doesn't really use pathlib because it's currently marked as provisional (or at least that's why I haven't tried to use it where possible in importlib). Do we have a plan of what is required to remove the provisional label from pathlib?

36 138

pathlib (was: Defining a path protocol)
by Jim J. Jewett April 10, 2016

April 10, 2016

(1) I think the "built-in" should instead be a module-level function in the pathlib. If you aren't already expecting pathlib paths, then you're just expecting strings to work anyhow, and a builtin isn't likely to be helpful. (2) I prefer that the function be explicit about the fact that it is downcasting the representation to a string. e.g., pathlib.path_as_string(my_path) But if the final result is ospath or fspath or ... I won't fight too hard, particularly since the output may be a bytestring rather than a str. -jJ

3 2

Incomplete Internationalization in Argparse Module
by Grady Martin April 9, 2016

April 9, 2016

Hello, all. I was wondering if the following string was left untouched by gettext for a purpose (from line 720 of argparse.py, in class ArgumentError): 'argument %(argument_name)s: %(message)s' There may be other untranslatable strings in the argparse module, but I have yet to encounter them in the wild. Thank you.

3 5

Re: [Python-Dev] Defining a path protocol (was: When should pathlib stop being provisional?)
by Koos Zevenhoven April 9, 2016

April 9, 2016

Nick Coghlan wrote: > On 7 April 2016 at 03:26, Brett Cannon <br...(a)python.org> wrote: >> >> Name: __path__, __fspath__, or something else? > > __fspath__ > I think I might like this dunder name because it does not clutter the list of regular methods and attributes, and is perhaps more pythonic. >> Method or attribute? (changes what kind of one-liner you might use in >> libraries, but I think historically all protocols have been methods and the >> serialized string representation might be costly to build) > > Method, as long as there's a helper function somewhere As a further minor benefit of it being a method, it may be easier to distinguish it from from `__path__`, which is an iterable attribute. >> Built-in? (name is dependent on #1 if we add one) > > os.fspath (alongside os.fsencode and os.fsdecode) > > (Putting this in a module low in the dependency stack makes it easy > for other modules to access without pulling in all of pathlib's > dependencies) Strong +1 on putting it in os. This should also be implemented in DirEntry, instances of which are "yielded" by os.scandir. Also, you have a strong case regarding naming with the 'fs' prefix. It is also easier to read fspath as f-s-path than it is to read ospath as o-s-path, because ospath could also be pronounced as a single (meaningless?) word. I'm still thinking a little bit about 'pathname', which to me sounds more like a string than fspath does [1]. It would be nice to have the string/path distinction especially when pathlib adoption grows larger. But who knows, maybe somewhere in the far future, no-one will care much about fspath, fsencode, fsdecode or os.path. >> Add the method/attribute to str? (I assume so, much like __index__() is on >> int, but I have not seen it explicitly stated so I would rather clarify it) > > Makes sense If added to str, it should also be added to bytes. But will that then return str or bytes? See also the next point. > Expand the C API to have something like PyObject_Path()? > > PyUnicode_FromFSPath, perhaps? The return type is well-defined here, > so it can be done as an alternate constructor, and the C API > counterparts of os.fsdecode and os.fsencode are PyUnicode functions > (specifically PyUnicode_DecodeFSDefault and PyUnicode_EncodeFSDefault) What about DirEntry, which may have a bytes representation? I would expect the function return type of os.fspath to be Union[str, bytes], unless bytes pathnames are decoded with surrogate escapes. [1] https://mail.python.org/pipermail/python-ideas/2016-April/039595.html PS. I have been reading this list occasionally on the google groups mirror, and I now subscribed to it just to send this. (BTW, I probably broke the thread, as I did not have Nick's email in my inbox to reply to. Sorry about that.) I'll have to mention that I was surprised, to say the least, to find that the pathlib discussion had moved here from python-ideas, where I had mentioned I was working on a proposal. Then, I also found that the solution discussed here was seemingly an improved version of what I had proposed on python-ideas somewhat earlier [1], but did not get any reactions to. While I can only make guesses about what happened, these kinds of things easily make you go from "Hey, maybe I'll be able to do something to improve Python!" to "These people don't seem to want me here or appreciate my efforts.". Not to accuse anyone in particular; just to let people know. Anyway, I somehow got sucked into thinking deeply about pathlib etc. (which I do use). Not that I really have much at stake here, except spending ridiculous amounts of time thinking about paths, mainly during my Easter holidays and after that. I really had a hard time explaining to friends and family what the heck I was doing ;).

6 9

Question about the current implementation of str
by Larry Hastings April 9, 2016

April 9, 2016

I have a straightforward question about the str object, specifically the PyUnicodeObject. I've tried reading the source to answer the question myself but it's nearly impenetrable. So I was hoping someone here who understands the current implementation could answer it for me. Although the str object is immutable from Python's perspective, the C object itself is mutable. For example, for dynamically-created strings the hash field may be lazy-computed and cached inside the object. I was wondering if there were other fields like this. For example, are there similar lazy-computed cached objects for the different encoded versions (utf8 utf16) of the str? What would really help an exhaustive list of the fields of a str object that may ever change after the object's initial creation. Thanks! We now return you to the debate about the pathlib module, //arry/

4 4

Summary of Python tracker Issues
by Python tracker April 8, 2016

April 8, 2016

ACTIVITY SUMMARY (2016-04-01 - 2016-04-08) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 5477 ( +6) closed 32993 (+22) total 38470 (+28) Open issues with patches: 2381 Issues opened (23) ================== #26686: email.parser stops parsing headers too soon. http://bugs.python.org/issue26686 opened by msapiro #26687: Use Py_RETURN_NONE in sqlite3 module http://bugs.python.org/issue26687 opened by berker.peksag #26689: Add `has_flag` method to `distutils.CCompiler` http://bugs.python.org/issue26689 opened by sylvain.corlay #26692: cgroups support in multiprocessing http://bugs.python.org/issue26692 opened by Satrajit Ghosh #26693: Exception ignored in: <module 'threading' from ...threading.py http://bugs.python.org/issue26693 opened by skydoom #26694: Disasembler fall with Key Error while disassemble obfuscated c http://bugs.python.org/issue26694 opened by pulina #26695: pickle and _pickle accelerator have different behavior when un http://bugs.python.org/issue26695 opened by josh.r #26696: Document collections.abc.ByteString http://bugs.python.org/issue26696 opened by brett.cannon #26697: tkFileDialog crash on askopenfilename Python 2.7 64-bit Win7 http://bugs.python.org/issue26697 opened by Eric Johnson #26698: Tk DPI awareness http://bugs.python.org/issue26698 opened by westley.martinez #26699: locale.str docstring is incorrect: "Convert float to integer" http://bugs.python.org/issue26699 opened by mark.dickinson #26700: Make digest_size a class variable http://bugs.python.org/issue26700 opened by rhettinger #26701: Documentation for int constructor mentions __int__ but not __t http://bugs.python.org/issue26701 opened by Robert Smallshire2 #26702: A better assert statement http://bugs.python.org/issue26702 opened by barry #26703: Socket state corrupts when original socket object goes out of http://bugs.python.org/issue26703 opened by JoshN #26704: unittest.mock.patch: Double patching instance method: Attribut http://bugs.python.org/issue26704 opened by asottile #26705: logging.Handler.handleError should be called from logging.Hand http://bugs.python.org/issue26705 opened by palaviv #26706: Update OpenSSL version in readme http://bugs.python.org/issue26706 opened by scw #26707: plistlib fails to parse bplist with 0x80 UID values http://bugs.python.org/issue26707 opened by slo.sleuth #26708: Constify C string pointers in the posix module http://bugs.python.org/issue26708 opened by serhiy.storchaka #26710: ConfigParser: Values in DEFAULT section override defaults pass http://bugs.python.org/issue26710 opened by Marc.Abramowitz #26711: Fix comparison of plistlib.Data http://bugs.python.org/issue26711 opened by serhiy.storchaka #26712: Unify (r)split(), (l/r)strip() method tests http://bugs.python.org/issue26712 opened by martin.panter Most recent 15 issues with no replies (15) ========================================== #26706: Update OpenSSL version in readme http://bugs.python.org/issue26706 #26700: Make digest_size a class variable http://bugs.python.org/issue26700 #26699: locale.str docstring is incorrect: "Convert float to integer" http://bugs.python.org/issue26699 #26697: tkFileDialog crash on askopenfilename Python 2.7 64-bit Win7 http://bugs.python.org/issue26697 #26696: Document collections.abc.ByteString http://bugs.python.org/issue26696 #26695: pickle and _pickle accelerator have different behavior when un http://bugs.python.org/issue26695 #26694: Disasembler fall with Key Error while disassemble obfuscated c http://bugs.python.org/issue26694 #26692: cgroups support in multiprocessing http://bugs.python.org/issue26692 #26672: regrtest missing in the module name http://bugs.python.org/issue26672 #26669: time.localtime(float("NaN")) does not raise a ValueError on al http://bugs.python.org/issue26669 #26667: Update importlib to accept pathlib.Path objects http://bugs.python.org/issue26667 #26665: pip is not bootstrapped by default on 2.7 http://bugs.python.org/issue26665 #26663: asyncio _UnixWritePipeTransport._close abandons unflushed writ http://bugs.python.org/issue26663 #26661: python fails to locate system libffi http://bugs.python.org/issue26661 #26660: tempfile.TemporaryDirectory() cleanup exception on Windows if http://bugs.python.org/issue26660 Most recent 15 issues waiting for review (15) ============================================= #26712: Unify (r)split(), (l/r)strip() method tests http://bugs.python.org/issue26712 #26711: Fix comparison of plistlib.Data http://bugs.python.org/issue26711 #26708: Constify C string pointers in the posix module http://bugs.python.org/issue26708 #26707: plistlib fails to parse bplist with 0x80 UID values http://bugs.python.org/issue26707 #26706: Update OpenSSL version in readme http://bugs.python.org/issue26706 #26705: logging.Handler.handleError should be called from logging.Hand http://bugs.python.org/issue26705 #26704: unittest.mock.patch: Double patching instance method: Attribut http://bugs.python.org/issue26704 #26689: Add `has_flag` method to `distutils.CCompiler` http://bugs.python.org/issue26689 #26687: Use Py_RETURN_NONE in sqlite3 module http://bugs.python.org/issue26687 #26685: Raise errors from socket.close() http://bugs.python.org/issue26685 #26680: Incorporating float.is_integer into the numeric tower and Deci http://bugs.python.org/issue26680 #26661: python fails to locate system libffi http://bugs.python.org/issue26661 #26658: test_os fails when run on Windows ramdisk http://bugs.python.org/issue26658 #26657: Directory traversal with http.server and SimpleHTTPServer on w http://bugs.python.org/issue26657 #26651: Deprecate register_adapter() and register_converter() in sqlit http://bugs.python.org/issue26651 Top 10 most discussed issues (10) ================================= #26680: Incorporating float.is_integer into the numeric tower and Deci http://bugs.python.org/issue26680 11 msgs #26693: Exception ignored in: <module 'threading' from ...threading.py http://bugs.python.org/issue26693 10 msgs #23551: IDLE to provide menu link to PIP gui. http://bugs.python.org/issue23551 9 msgs #26689: Add `has_flag` method to `distutils.CCompiler` http://bugs.python.org/issue26689 9 msgs #26703: Socket state corrupts when original socket object goes out of http://bugs.python.org/issue26703 9 msgs #18844: allow weights in random.choice http://bugs.python.org/issue18844 8 msgs #24291: wsgiref.handlers.SimpleHandler truncates large output blobs http://bugs.python.org/issue24291 7 msgs #25609: Add a ContextManager ABC and type http://bugs.python.org/issue25609 7 msgs #26257: Eliminate buffer_tests.py http://bugs.python.org/issue26257 7 msgs #26707: plistlib fails to parse bplist with 0x80 UID values http://bugs.python.org/issue26707 7 msgs Issues closed (21) ================== #6953: readline documentation needs work http://bugs.python.org/issue6953 closed by martin.panter #10796: Improve doc for readline.set_completer_delims() http://bugs.python.org/issue10796 closed by martin.panter #23371: mimetypes initialization fails on Windows because of TypeError http://bugs.python.org/issue23371 closed by berker.peksag #23735: Readline not adjusting width after resize with 6.3 http://bugs.python.org/issue23735 closed by martin.panter #25951: SSLSocket.sendall() does not return None on success like socke http://bugs.python.org/issue25951 closed by martin.panter #25987: collections.abc.Reversible http://bugs.python.org/issue25987 closed by gvanrossum #26234: The typing module includes 're' and 'io' in __all__ http://bugs.python.org/issue26234 closed by gvanrossum #26295: Random failures when running test suite in parallel (-m test - http://bugs.python.org/issue26295 closed by haypo #26391: typing: Specialized sub-classes of Generic never call __init__ http://bugs.python.org/issue26391 closed by gvanrossum #26479: Init documentation typo "may be return" > "may NOT be returned http://bugs.python.org/issue26479 closed by Samuel Colvin #26509: asyncio: spurious ConnectionAbortedError logged on Windows http://bugs.python.org/issue26509 closed by haypo #26586: Simple enhancement to BaseHTTPRequestHandler http://bugs.python.org/issue26586 closed by martin.panter #26671: Clean up path_converter in posixmodule.c http://bugs.python.org/issue26671 closed by serhiy.storchaka #26673: Tkinter error when opening IDLE configuration menu http://bugs.python.org/issue26673 closed by terry.reedy #26678: Incorrect linking to elements in datetime package http://bugs.python.org/issue26678 closed by martin.panter #26679: curses: Descripton of KEY_NPAGE and KEY_PPAGE inverted http://bugs.python.org/issue26679 closed by berker.peksag #26688: unittest2 referenced in unittest.mock documentation http://bugs.python.org/issue26688 closed by berker.peksag #26690: PyUnicode_Decode breaks when Python / sqlite3 is built with sq http://bugs.python.org/issue26690 closed by zzzeek #26691: Update the typing module to match what's in github.com/python/ http://bugs.python.org/issue26691 closed by gvanrossum #26709: Year 2038 problem in plistlib http://bugs.python.org/issue26709 closed by serhiy.storchaka #26713: Change f-literal grammar so that escaping isn’t possible or http://bugs.python.org/issue26713 closed by r.david.murray

1 0

Other pathlib improvements? was: When should pathlib stop being provisional?
by Eric Snow April 8, 2016

April 8, 2016

On Apr 6, 2016 11:11 PM, "Raymond Hettinger" <raymond.hettinger(a)gmail.com> wrote: > Having worked through the API when it is first released, I find it to be highly forgettable (i.e. I have to re-read the docs each time I've revisited it). Agreed, though it's arguably better than argparse, logging, unittest, or several other stdlib modules. To some extent the challenge with those is the complexity of the problem space. Furthermore, the key for any sufficiently complex module is that the common-case usage is intuitive and simple enough. Some stdlib modules do a better job of that than others. :/ How much would you say that any of that applies to pathlib? What about relative to other similar packages on the cheeseshop? Regardless, are there any specific improvements you'd recommend while the module is still provisional? Are your concerns a matter of structure vs. naming? Usability vs. (intuitive) discoverability? -eric

7 7

summary: a Path protocol
by Ethan Furman April 8, 2016

April 8, 2016

The discussion has ranged all over, so let me try to sum up: Name: __ospath__ Method or attribute? Method (implementations are of course free to pre-build and/or cache the value) Built-in? no, rather a function in pathlib - ospath() Add the method/attribute to str? Not necessary -- but if somebody else wants to do that part I am not opposed Expand the C API to have something like PyObject_Path()? Yes - and if I understood correctly this function will do the same as pathlib.ospath(), just at the C level? And what will its name be, exactly? -- ~Ethan~

3 2