Python-ideas September 2015

python-ideas@python.org

107 participants
54 discussions

PEP 504: Using the system RNG by default

by Nick Coghlan

Hi folks, Based on the feedback in the recent threads, I've written a draft PEP that dispenses with the userspace CSPRNG idea, and instead proposes: * defaulting to using the system RNG for the module level random API in Python 3.6+ * implicitly switching to the deterministic PRNG if you call random.seed(), random.getstate() or random.setstate() (this implicit fallback would trigger a silent-by-default deprecation warning in 3.6, and a visible-by-default runtime warning after 2.7 goes EOL) * providing random.system and random.seedable submodules so you can explicitly opt in to using the one you want without having to manage your own RNG instances That approach would provide a definite security improvement over the status quo, while restricting the compatibility break to a performance regression in applications that use the module level API without calling seed(), getstate() or setstate(). It would also allow the current security warning in the random module documentation to be moved towards the end of the module, in a section dedicated to determinism and reproducibility. The full PEP should be up shortly at https://www.python.org/dev/peps/pep-0504/, but caching is still a problem when uploading new PEPs, so if that 404s, try http://legacy.python.org/dev/peps/pep-0504/ Regards, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

4 years, 8 months

Using "||" (doubled pipe) as the null coalescing operator?

by Nick Coghlan

This may just be my C programmer brain talking, but reading the examples in PEP 505 makes me think of the existing use of "|" as the bitwise-or operator in both Python and C, and "||" as the logical-or operator in C. Using || for None-coalescence would still introduce a third "or" variant into Python as PEP 505 proposes (for good reasons), but without introducing a new symbolic character that relates to "OR" operations: x | y: bitwise OR (doesn't short circuit) x or y: logical OR (short circuits based on bool(x)) x || y: logical OR (short circuits based on "x is not None") (An analogy with C pointers works fairly well here, as "x || y" in C is a short-circuiting operator that switches on "x != NULL" in the pointer case) Taking some key examples from the PEP: data = data ?? [] headers = headers ?? {} data ?= [] headers ?= {} When written using a doubled pipe instead: data = data || [] headers = headers || {} data ||= [] headers ||= {} Translations would be the same as proposed n PEP 505 (for simplicity, this shows evaluating the LHS multiple times, in practice that wouldn't happen): data = data if data is not None else [] headers = headers if headers is not None else [] data = data if data is not None else [] headers = headers if headers is not None else [] One additional wrinkle is that a single "|" would conflict with the bitwise-or notation in the case of None-aware index access, so the proposal for both that and attribute access would be to make the notation "!|", borrowing the logical negation "!" from "!=". In this approach, where "||" would be the new short-circuiting binary operator standing for "LHS if LHS is not None else RHS", in "!|" the logical negations cancel out to give "LHS if LHS is None else LHS<OP>". PEP 505 notation: title?.upper() person?['name'] Using the "is not not None" pipe-based notation: title!|.upper() person!|['name'] And the gist of the translation: title if title is None else title.upper() person if person is None else person['name'] If this particular syntax were to be chosen, I also came up with the following possible mnemonics that may be useful as an explanatory tool: "||" is a barrier to prevent None passing through an expression "!|" explicitly allows None to pass without error Regards, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

4 years, 8 months

Null coalescing operators

by Mark Haase

StackOverflow has many questions <http://stackoverflow.com/search?q=%5Bpython%5D+null+coalesce> on the topic of null coalescing operators in Python, but I can't find any discussions of them on this list or in any of the PEPs. Has the addition of null coalescing operators into Python ever been discussed publicly? Python has an "or" operator that can be used to coalesce false-y values, but it does not have an operator to coalesce "None" exclusively. C# has nice operators for handling null: "??" (null coalesce), "?." (null-aware member access), and "?[]" (null-aware index access). They are concise and easy to reason about. I think these would be a great addition to Python. As a motivating example: when writing web services, I often want to change the representation of a non-None value but also need to handle None gracefully. I write code like this frequently: response = json.dumps({ 'created': created.isoformat() if created is not None else None, 'updated': updated.isoformat() if updated is not None else None, ... }) With a null-aware member access operator, I could write this instead: response = json.dumps({ 'created': created?.isoformat(), 'updated': updated?.isoformat(), ... }) I can implement this behavior myself in pure Python, but it would be (a) nice to have it the in the standard library, and (b) even nicer to have an operator in the language, since terseness is the goal. I assume that this has never been brought up in the past because it's so heinously un-Pythonic that you'd have to be a fool to risk the public mockery and shunning associated with asking this question. Well, I guess I'm that fool: flame away... Thanks, Mark

4 years, 8 months

Re: [Python-ideas] Null coalescing operators

by Chris Barker - NOAA Federal

Sent from my iPhone On Sep 22, 2015, at 6:21 PM, Fernando Perez <fperez.net(a)gmail.com In fact, that already happened once: before Python 2.4, our prefix for "magic functions" was the @ character, and when that was introduced as the decorator prefix, we had to scramble.� Note that the userbase of iPython was orders of magnitude smaller then -- changes like that would be a much bigger deal now. And while iPython was born in the scientific software community, and sees a lot (most) of its use there, it is by no means specific to that use case. In fact, if you use the standard REPL at all -- I encourage you to give it a try -- you will be very glad you did. -Chris

4 years, 8 months

Re: [Python-ideas] add a single __future__ for py3?

by Chris Barker - NOAA Federal

On Sep 22, 2015, at 6:43 PM, Andrew Barnert <abarnert(a)yahoo.com> wrote: On Sep 21, 2015, at 10:59, Gregory P. Smith <greg(a)krypto.org> wrote: I think people should stick with *from __future__ import absolute_import* regardless of what code they are writing. If the py3 way of handling Absolute vs relative import isn't better --- why is it in Py3???? Anyway, the point of a this is to get your py2 code working as similarly as possible on py3. So better or worse, or not all that different, you still want that behavior. But again, it looks like this ship has sailed... Thanks for indulging me. -Chris They will eventually create a file innocuously called something like calendar.py (the same name as a standard library module) in the same directory as their main binary and their debugging of the mysterious failures they just started getting from the tarfile module will suddenly require leveling up to be able to figure it out. ;) But they'll get the same problems either way. If calendar.py isn't on sys.path, it won't interfere with tarfile. And if it is on sys.path, so it does interfere with tarfile, then it's already an absolute import, so enabling absolute_import doesn't help. I suppose if they've done something extra stupid, like putting a package directory on sys.path as well as putting something called calendar.py in that package and importing it with an unqualified import, then maybe it'll be easier for someone to explain all the details of everything they did wrong (including why they shouldn't have put the package on sys.path) if they're using absolute_imports, but beyond that, I don't see how it helps this case. -gps On Mon, Sep 21, 2015 at 8:18 AM Guido van Rossum <guido(a)python.org> wrote: > It's just about these four imports, right? > > > from __future__ import absolute_import > from __future__ import division > from __future__ import print_function > from __future__ import unicode_literals > > I think the case is overblown. > > - absolute_import is rarely an issue; the only thing it does (despite the > name) is give an error message when you attempt a relative import without > using a "." in the import. A linter can find this easily for you, and a > little discipline plus the right example can do a lot of good here. > > - division is important. > > - print_function is important. > > - unicode_literals is useless IMO. It breaks some things (yes there are > still APIs that don't take unicode in 2.7) and it doesn't nearly as much as > what would be useful -- e.g. repr() and <stream>.readline() still return > 8-bit strings. I recommend just using u-literals and abandoning Python 3.2. > > -- > --Guido van Rossum (python.org/~guido) > _______________________________________________ > Python-ideas mailing list > Python-ideas(a)python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ _______________________________________________ Python-ideas mailing list Python-ideas(a)python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

4 years, 8 months

add a single __future__ for py3?

by Chris Barker

Hi all, the common advise, these days, if you want to write py2/3 compatible code, is to do: from __future__ import absolute_import from __future__ import division from __future__ import print_function from __future__ import unicode_literals https://docs.python.org/2/howto/pyporting.html#prevent-compatibility-regres… I'm trying to do this in my code, and teaching my students to do it to. but that's actually a lot of code to write. It would be nice to have a: from __future__ import py3 or something like that, that would do all of those in one swipe. IIIC, l can't make a little module that does that, because the __future__ imports only effect the module in which they are imported Sure, it's not a huge deal, but it would make it easier for folks wanting to keep up this best practice. Of course, this wouldn't happen until 2.7.11, if an when there even is one, but it would be nice to get it on the list.... -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

4 years, 8 months

Re: [Python-ideas] [Python-Dev] Make stacklevel=2 by default in warnings.warn()

by Steven D'Aprano

On Sat, Sep 19, 2015 at 11:55:44PM -0700, Nathaniel Smith wrote: > I don't have enough fingers to count how many times I've had to > explain how stacklevel= works to maintainers of widely-used packages > -- they had no idea that this was even a thing they were getting > wrong. Count me in that. I had no idea it was even a thing. -- Steve

4 years, 8 months

new format spec for iterable types

by Wolfgang Maier

Hi, in the parallel "format and print" thread, Andrew Barnert wrote: > For example, in a 5-line script I wrote last night, I've got > print(head, *names, sep='\t'). I could have used > print('\t'.join(chain([head], names)) instead--in fact, any use of > multi-argument print can be replaced by > print(sep.join(map(str, args)))--but that's less convenient, less > readable, and less likely to occur to novices. And there are plenty > of other alternatives, from > print('{}\t{}'.format(head, '\t'.join(names)) to ... That last thing, '{}\t{}'.format(head, '\t'.join(names)), is something I find myself writing relatively often - when I do not want to print the result immediately, but store it - but it is ugly to read with its nested method calls and the separators occurring in two very different places. Now Andrew's comment has prompted me to think about alternative syntax for this and I came up with this idea: What if built in iterable types through their __format__ method supported a format spec string of the form "*separator" and interpreted it as join your elements' formatted representations using "separator" ? A quick and dirty illustration in Python: class myList(list): def __format__ (self, fmt=''): if fmt == '': return str(self) if fmt[0] == '*': sep = fmt[1:] or ' ' return sep.join(format(e) for e in self) else: raise TypeError() head = 99 data = myList(range(10)) s = '{}, {:*, }'.format(head, data) # or s2 = '{}{sep}{:*{sep}}'.format(head, data, sep=', ') print(s) print(s2) # 99, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 Thoughts?

4 years, 8 months

Structural type checking for PEP 484

by Guido van Rossum

Jukka wrote up a proposal for structural subtyping. It's pretty good. Please discuss. https://github.com/ambv/typehinting/issues/11#issuecomment-138133867 -- --Guido van Rossum (python.org/~guido)

4 years, 8 months

Bring line continuation to multi-level dictionary lookup

by John Wong

Hi everyone. I work with APIs which have deep nested dictionary structure response. Imagine a simplified case: foo = {1: {2: {3: {4: {5: 6 } } } } Now imagine I need to get to 6: foo['1']['2']['3']['4']['5']['6'] This looks managable, but if the key name is long, then I certainly will end doing this to respect my style guide. To make it concrete, let's use something reallistic, a response call from AWS API: response = {'DescribeDBSnapshotsResponse': {'ResponseMetadata': {'RequestId': '123456'}, 'DescribeDBSnapshotsResult': {'Marker': None, 'DBSnapshots': [{'Engine': 'postgres'}]}}} If I had to get to the Engine I'd do this: detail_response = response["DescribeDBSnapshotsResponse"] result = detail_response["DescribeDBSnapshotsResult"] This is only a few level deep, but imagine something slightly longer (I strict out so much from this response). Obviously I am picking some real example but key name being really long to sell my request. Can we do it differently? How about print(response.get( "DescribeDBSnapshotsResponse").get( "DescribeDBSnapshotsResult").get( "DBSnapshots")[0].get( "Engine")) Okay. Not bad, almost like writing in Javascript except Python doesn't allow you to do line continuation before the got at all, so you are stuck with (. But the problem with the alternative is that if DescribeDBSnapshotsResult is a non-existent key, you will just get None, because that's the beauty of the .get method for a dictionary object. So while this allows you to write in slightly different way, I am costing silent KeyError exception. I wouldn't know which key raised the exception. Whereas with [key1][key2] I know if key1 doesn't exist, the exception will explain to me that key1 does not exist. So here I am, thinking, what if we can do this? response( ["DescribeDBSnapshotsResponse"] ["DescribeDBSnapshotsResult"] ) You get the point. This looks kinda ugly, but it doesn't require so many assignment. I think this is doable, after all [ ] is still a method call with the key name passed in. I am not familar with grammar, so I don't know how hard and how much the implementation has to change to adopt this. Let me know if this is a +1 or -10000000 bad crazy idea. Thanks. John

4 years, 8 months

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas September 2015