Null coalescing operator

Sometimes I find myself in need of this nice operator that I used back in the days when I was programming in .NET, essentially an expression
expr ?? instead
should return expr when it `is not None` and `instead` otherwise. A piece of code that I just wrote, you can see a use case: def _sizeof(self, context): if self.totalsizeof is not None: return self.totalsizeof else: raise SizeofError("cannot calculate size") With the oprator it would just be def _sizeof(self, context): return self.totalsizeof ?? raise SizeofError("cannot calculate size") pozdrawiam, Arkadiusz Bulski

On Sat, Sep 10, 2016 at 6:01 AM, Arek Bulski <arek.bulski@gmail.com> wrote:
You can use 'or' for this, as long as you're okay with other falsey values being treated the same way. In a lot of cases, this isn't a problem. However, even if this is implemented, it would be in an expression context, so 'raise' would never work. For that, I'd just use the explicit statement form. ChrisA

On Fri, Sep 9, 2016 at 3:01 PM, Arek Bulski <arek.bulski@gmail.com> wrote:
This was proposed almost exactly a year ago, start reading here: https://mail.python.org/pipermail/python-ideas/2015-September/036289.html -- Zach

This idea has come up before. While I can see the use of it, to me at least that use doesn't feel nearly common enough to warrant dedicated syntax. In many cases, it is a "truthy" value you are looking for rather than `is not None` specifically. That has a convenient spelling: expr or instead If it really is the actual None-ness you are curious about, you need the slightly longer: expr if expr is not None else instead Your example seems to want to fall back to a statement suite rather than a value. To do that, you'd have to put the suite inside a function such as: def Raise(err): raise err And use it something like: self.totalsizeof or Raise(SizeofError(...)) On Fri, Sep 9, 2016 at 1:01 PM, Arek Bulski <arek.bulski@gmail.com> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

I'd note you can also save 4 characters by writing: instead if expr is None else expr On Fri, Sep 9, 2016 at 1:10 PM, David Mertz <mertz@gnosis.cx> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On 2016-09-09 21:01, Arek Bulski wrote:
'raise' is a statement, so it can't appear in an expression. This has been discussed before, so you might want to read this thread first: Null coalescing operators https://mail.python.org/pipermail/python-ideas/2015-September/036289.html

On Fri, Sep 09, 2016 at 10:01:44PM +0200, Arek Bulski wrote:
As Zach and MRAB mention, this was discussed last year. If I recall correctly, the discussion fizzled out without a solid conclusion. I think there's a PEP -- if not, there should be. I would be interested in revisiting this idea, but 3.6 feature freeze is only a day or two away and I won't have time to discuss this before then. So let's please drop this discussion until the 3.6 beta is released. -- Steve

Steven D'Aprano writes:
The problem, as I recall, was that there was a conclusion that 'maybe_None if maybe_None is None else default' and the less precise 'maybe_falsy or default' are sufficient if you just want to provide a default if a value is "null". The proponents pointed out that other operators would benefit from null coalescing, and then discussion petered out because there was no remotely plausible suggestion except to use '?' to indicate null-coalescing versions. But Guido has historically resisted any use of '?' whatsoever in Python syntax, and there weren't any good alternatives or strong proponents of ?-based syntax. I forget if Guido was very sympathetic to null-coalescing operators, given somebody came up with a good syntax. There was also an issue of whether SQL NULLs and similar constructs "should" be spelled None in Python, and if not, how would null coalescence be defined. Steve

On Sat, Sep 10, 2016, at 12:48, Stephen J. Turnbull wrote:
I forget if Guido was very sympathetic to null-coalescing operators, given somebody came up with a good syntax.
As I remember the discussion, I thought he'd more or less conceded on the use of ? but there was disagreement on how to implement it that never got resolved. Concerns like, you can't have a?.b return None because then a?.b() isn't callable, unless you want to use a?.b?() for this case, or some people wanted to have "a?" [where a is None] return a magic object whose attribute/call/getitem would give no error, but that would have to keep returning itself and never actually return None for chained operators.

The way I recall it, we arrived at the perfect syntax (using ?) and semantics. The issue was purely strong hesitation about whether sprinkling ? all over your code is too ugly for Python, and in the end we couldn't get agreement on *that*. Another problem is PEP 505 -- it is full of discussion but its specification is unreadable due to the author's idea to defer the actual choice of operators and use a strange sequence of unicode characters instead. If someone wants to write a new, *short* PEP that defers to PEP 505 for motivation etc. and just writes up the spec for the syntax and semantics we'll have a better starting point. IMO the key syntax is simply one for accessing attributes returning None instead of raising AttributeError, so that e.g. `foo?.bar?.baz` is roughly equivalent to `foo.bar.baz if (foo is not None and foo.bar is not None) else None`, except evaluating foo and foo.bar only once. On Sat, Sep 10, 2016 at 10:14 AM, Random832 <random832@fastmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)

On 10 September 2016 at 18:26, Guido van Rossum <guido@python.org> wrote:
If we're not looking to use all the other null-coalescing variants (?=, ?(), ...) - which is something I'm pleased about, as I do think that scattering that many ?'s about is likely to lead to ugly code - then it would probably be fine to just use ? for this operation, so we'd have foo?bar?baz rather than needing foo?.bar?.baz. Paul

On Sat, Sep 10, 2016 at 11:09 AM, MRAB <python@mrabarnett.plus.com> wrote:
Indeed. And ?. is how this is spelled in some other lanuages (C# and Dart). I forgot one detail that's in PEP 505: e.g. `foo?.bar.baz()` should be implemented as `foo.bar.baz() if foo is not None else None`. IOW if foo is None, the entire trailing section `.bar.baz()` should be skipped. (But this is a property of `?.` as an alternative attribute access operator; it doesn't mean `?` is a postfix operator on `foo`.) Another issue already discussed in PEP 505 is a conflict with IPython (Jupyter Notebook), which uses ? and ?? as custom syntax to request help. But maybe it can be taught to only recognize those when they're the last character(s) on the line? -- --Guido van Rossum (python.org/~guido)

On Sat, Sep 10, 2016 at 4:56 PM, Guido van Rossum <guido@python.org> wrote:
I think this is already the case: In [1]: ?foo Object `foo` not found. In [2]: foo? Object `foo` not found. In [3]: foo?bar File "<ipython-input-3-c7de2a05ce6b>", line 1 foo?bar ^ SyntaxError: invalid syntax

On Sat, Sep 10, 2016, at 13:26, Guido van Rossum wrote:
I think that if there's "strong hesitation" about something being "too ugly" it can't really be described as "the perfect syntax". IIRC there were a couple alternatives being discussed that would have reduced the number of question marks to one [or one per object which might be None].

I find the '?.' syntax very ugly, much more so in the examples of chained attributes. A much better way to handle the use case is to wrap objects in a class that gives this "propagating None" behavior with plain attribute access. A nice implementation was presented in this thread. On Sep 10, 2016 3:16 PM, "Random832" <random832@fastmail.com> wrote:

https://github.com/kirbyfan64/_frozensafemockobjectimplementation In all seriousness, though, I really feel like that would be the ultimate bug magnet, since it'd be easy to forget to un-wrap the object afterwards. -- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/ On Sep 10, 2016 5:21 PM, "David Mertz" <mertz@gnosis.cx> wrote:

Actually, I guess the example I liked was from the year ago discussion. And it didn't do *exactly* what I think a wrapper should. What I'd want would be like this: class NoneCoalesce(object): "Standard operations on object for 'is not None'" def __init__(self, obj): self.obj = obj def __getattr__(self, name): try: return getattr(self.obj, name) except AttributeError: return NoneCoalesce(None) def __getitem__(self, item): try: return self.obj[item] except (TypeError, KeyError): return NoneCoalesce(None) def __call__(self, *args, **kwds): try: return self.obj(*args, **kwds) except TypeError: return NoneCoalesce(None) def __bool__(self): return self.obj is not None def __repr__(self): return "NoneCoalesce[%r]" % self.obj def __str__(self): return "NoneCoalesce[%r]" % self.obj def __len__(self): try: return len(self.obj) except TypeError: return 0 Then we might use it similar to this:
Nothing special about boltons' OrderedMultiDict here, just something I've been playing with that has some distinctive methods. The idea is that we can easily have both "regular" behavior and None coalescing just by wrapping any objects in a utility class... and WITHOUT adding ugly syntax. I might have missed some corners where we would want behavior wrapped, but those shouldn't be that hard to add in principle. On Sat, Sep 10, 2016 at 3:21 PM, David Mertz <mertz@gnosis.cx> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On Sun, Sep 11, 2016 at 9:10 AM, Guido van Rossum <guido@python.org> wrote:
So you're offering `NoneCoalesce(x).bar` as less-ugly alternative to `x?.bar`... Color me unconvinced.
As a syntactic form? Not interested. But what if it's the underlying implementation? We have "yield from X" as a tidy syntax for roughly a page of equivalent code. We could have "x?.bar" as syntactic sugar for "NoneCoalesce(x).bar". ChrisA

On Sat, Sep 10, 2016 at 4:27 PM, Chris Angelico <rosuav@gmail.com> wrote:
PEP 505 has an option for a way to customize the coalescing operation (https://www.python.org/dev/peps/pep-0505/#generalized-coalescing). Though I think I'd rather not do that. But it just occurs to me that the implementation given by David Mertz is not what I'd expect: it seems that `NoneCoalesce([]).flup` would catch the AttributeError (there' no `[].flup`) and return NoneCoalesce(None), whereas I would expect `?.` to only return None when the LHS is None, not when some other not-None object doesn't have the requested attribute. (And the "pile of poo" operator in PEP 505 agrees with me.) -- --Guido van Rossum (python.org/~guido)

(Replying to multiple posts in this thread) Guido van Rossum:
Hi, I wrote PEP-505. I'm sorry that it's unreadable. The choice of emoji as operators was supposed to be a blatant joke. I'd be happy to submit a new version that is ASCII. Or make any other changes that would facilitate making a decision on the PEP. As I recall, the thread concluded with Guido writing, "I'll have to think about this," or something to that effect. I had hoped that the next step could be a survey where we could gauge opinions on the various possible spellings. I believe this was how PEP-308 was handled, and that was a very similar proposal to this one. Most of the discussion on list was really centered around the fact that nobody like the proposed ?? or .? spellings, and nobody could see around that fact to consider whether the feature itself was intrinsically valuable. (This is why the PEP doesn't commit to a syntax.) Also, as unfortunate side effect of a miscommunication, about 95% of the posts on this PEP were written _before_ I submitted a complete draft and so most of the conversation was arguing about a straw man. David Mertz:
The biggest problem with a wrapper in practice is that it has to be unwrapped before it can be passed to any other code that doesn't know how to handle it. E.g. if you want to JSON encode an object, you need to unwrap all of the NullCoalesce objects because the json module wouldn't know what to do with them. The process of wrapping and unwrapping makes the resulting code more verbose than any existing syntax.
There's a script in the PEP-505 repo that can you help you identify code that could be written with the proposed syntax. (It doesn't identify blocks that would not be affected, so this doesn't completely answer your question.) https://github.com/mehaase/pep-0505/blob/master/find-pep505.py The PEP also includes the results of running this script over the standard library. On Sat, Sep 10, 2016 at 1:26 PM, Guido van Rossum <guido@python.org> wrote:

I actually think the spelling is the main stumbling block. The intrinsic value of the behavior is clear, it's finding an acceptable spelling that hold back the proposal. I propose that the next phase of the process should be to pick the best operator for each sub-proposal. Then we can decide which of the sub-proposals we actually want in the language, based on a combination of how important the functionality is and how acceptable we find the spelling. --Guido On Thu, Oct 13, 2016 at 8:20 PM, Mark E. Haase <mehaase@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)

For what it's worth, I like the C# syntax with question marks. It is probably more risky (breaks more code) to introduce a new keyword than a new symbol as operator. If we have to pick a symbol, it's less confusing if we pick something another language already uses. There is no shame in copying from other languages. Many of them copy ideas from Python as well ;-) Thanks. On 14 October 2016 at 17:10, Guido van Rossum <guido@python.org> wrote:
-- Gustavo J. A. M. Carneiro Gambit Research "The universe is always one step beyond logic." -- Frank Herbert

On 15 October 2016 at 13:36, Guido van Rossum <guido@python.org> wrote:
Having been previously somewhere between -1 and -0, I've been doing a lot more data mining and analysis work lately, which has been enough to shift me to at least +0 and potentially even higher when it comes to the utility of adding these operators (more on that below). = Pragmatic aspects = Regarding the spelling details, my current preferences are as follows: * None-coalescing operator: x ?or y * None-severing operator: x ?and y * None-coalescing augmented assignment: x ?= y * None-severing attribute access: x?.attr * None-severing subscript lookup: x?[expr] (The PEP currently only covers the "or?" and "and?" operator spelling suggestions, but the latter three suggestions are the same as those in the current PEP draft) My rationale for this preference is that it means that "?" is consistently a pseudo-operator that accepts an expression on the left and another binary operator (from a carefully restricted subset) on the right, and the combination is a new short-circuiting binary operation based on "LHS is not None". The last three operations can be defined in terms of the first two (with the usual benefit of avoiding repeated evaluation of the subexpression): * None-coalescing augmented assignment: x = x ?or y * None-severing attribute access: x ?and x.attr * None-severing subscript lookup: x ?and x[expr] The first two can then be defined in terms of equivalent if/else statements containing an "x is not None" clause: * None-coalescing operator: x if x is not None else y * None-severing operator: y if x is not None else x Importantly, the normal logical and/or can be expanded in terms of if/else in exactly the same way, only using "bool(x)" instead of "x is not None": * Logical or: x if x else y * Logical and: y if x else x = Language design philosophy aspects = Something I think is missing from the current PEP is a high level explanation of the *developer problem* that these operators solve - while the current PEP points to other languages as precedent, that just prompts the follow on question "Well, why did *they* add them, and does their rationale also apply to Python?". Even the current motivating examples don't really cover this, as they're quite tactical in nature ("Here is how this particular code is improved by the proposed change"), rather than explaining the high level user benefit ("What has changed in the surrounding technology environment that makes us think this is a user experience design problem worth changing the language definition to help address *now* even though Python has lived happily without these operators for 25+ years?") With conditional expressions, we had the clear driver that folks were insisting on using (and teaching!) the "and/or" hack as a workaround, and introducing bugs into their code as a result, whereas we don't have anything that clear-cut for this proposal (using "or" for None-coalescing doesn't seem to be anywhere near as popular as "and/or" used to be as an if/else equivalent). My point of view on that is that one of the biggest computing trends in recent years is the rise of "semi-structured data", where you're passing data around in either JSON-compatible data structures, or comparable structures mapped to instances and attributes, and all signs point to that being a permanent state change in the world of programming rather than merely being a passing fad. The world itself is fuzzy and ambiguous, and learning to work effectively with semi-structured data better reflects that ambiguity rather than forcing a false precision for the sake of code simplification. When you're working in that kind of context, encountering "None" is typically a shorthand for "This entire data subtree is missing, so don't try to do anything with it", but if it *isn't* None, you can safely assume that all the mandatory parts of that data segment will be present (no matter how deeply nested they are). To help explain that, it would be useful to mention not only the corresponding operators in other languages, but also the changes in data storage practices, like PostgreSQL's native support for JSON document storage and querying ( https://www.postgresql.org/docs/9.4/static/functions-json.html ) as well as the emergence/resurgence of hierarchical document storage techniques and new algorithms for working with them. However, it's also the case that where we *do* have a well understood and nicely constrained problem, it's still better to complain loudly when data is unexpectedly missing, rather than subjecting ourselves to the pain of having to deal with detecting problems with our data far away from where we introduced those problems. A *lot* of software still falls into that category, especially custom software written to meet the needs of one particular organisation. My current assumption is that those of us that now regularly need to deal with semi-structured data are thinking "Yes, these additions are obviously beneficial and improve Python's expressiveness, if we can find an acceptable spelling". Meanwhile, folks dealing primarily with entirely structured or entirely unstructured data are scratching their heads and asking "What's the big deal? How could it ever be worth introducing more line noise into the language just to make this kind of code easier to write?" Even the PEP's title is arguably a problem on that front - "None-aware operators" is a proposed *solution* to the problem of making semi-structured data easier to work with in Python, and hence reads like a solution searching for a problem to folks that don't regularly encounter these issues themselves. Framing the problem that way also provides a hint on how we could *document* these operations in the language reference in a readily comprehensible way: "Operators for working with semi-structured data" Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 15.10.2016 08:10, Nick Coghlan wrote:
Definitely true. Stricter rules are similar to "fail early", "no errors should pass silently" and the like. This stance is conveyed by Python as long as I know it.
That's where I like to see a common middle ground between those two sides of the table. I need to work with both sides for years now. In my experience, it's best to avoid semi-structured data at all to keep the code simple. As we all know and as you described, the world isn't perfect and I can only agree. However, what served us best in recent years, is to keep the "semi-" out of the inner workings of our codebase. So, handling "semi-" at the system boundary proved to be a reliable way of not breaking everything and of keeping our devs sane. I am unsure how to implement such solution, whether via PEP8 or via the proposal's PEP. It somehow reminds me of the sans-IO idea where the core logic should be simple/linear code and the difficult/problematic issues are solved at the systems boundary. This said, let me put it differently by using an example. I can find None-aware operators very useful at the outermost function/methods of a process/library/class/module: class FanzyTool: def __init__(self, option1=None, option2=None, ...): # what happens when option6 and option7 are None # and it only matters when option 3 is not None # but when ... Internal function/methods/modules/classes and even processes/threads should have a clear, non-wishy-washy way of input and output (last but not least also to do unit-testing on relatively sane level). def _append_x(self, s): return s + 'x' # strawman operation Imagine, that s is something important to be passed around many times inside of "FanzyTool". The whole process usually makes no sense at all, when s is None. And having each internal method checking for None is getting messy fast. I hope we can also convey this issue properly when we find an appropriate syntax.
That's indeed an extraordinarily good title as it describes best what we intend it to be used for (common usage scenarios). +1 Regards, Sven

On Oct 15, 2016, at 04:10 PM, Nick Coghlan wrote:
I'm sympathetic to (some of) the goals of PEP 505, as these issues do occasionally annoy me. But I'm not entirely convinced they are common enough or annoying enough to warrant special syntax, and I *really* dislike the introduction of a ? operator for these purposes. I'm also concerned about adopting too much generality muddling up what I think should be a narrowly targeted improvement to readability. The other thing to note is that, while I often use ternary operators for this now, checking against None isn't always the sole conditional. E.g. self.chain = (chain if chain is None or IChain.providedBy(chain) else config.chains[chain]) That being said, null-aware member access (NAMA) would be pretty handy occasionally. I'm less sure about the other forms. For me, the biggest benefit of NAMA is the short-circuiting of chained attribute access. I don't like the operator syntax because I find it less readable (harder for the eye to pick out), and because it isn't a completely obvious operation. But also because I generally want to chase the attributes all-or-nothing. For example, foo.bar.baz.qux but only if all the intermediary attributes resolve to non-Nones. I don't want to have to write foo.?bar.?baz.?qux I tried playing around with new keywords such as 'when' and 'unless', which seem a little nice although not a perfect fit. thing = foo.bar.baz.qux unless None thing = unless None then foo.bar.baz.qux thing = when foo.bar.baz.qux thing = foo.bar.baz.qux when not None I do like the idea of a keyword more than an operator, and disagree that a new keyword can't be introduced until Python 4. That's why we have __future__! Anyway, that's my $0.02. I trust Guido to DTPT (do the Pythonic thing :), even if that means rejecting the PEP. Cheers, -Barry

Am Freitag, 14. Oktober 2016 23:11:48 UTC-7 schrieb Nick Coghlan:
This is, more or less, the syntax added in Nick's PEP 531 draft <https://www.python.org/dev/peps/pep-0531/>. The reddit discussion <https://www.reddit.com/r/Python/comments/59d2he/pep_531_existence_checking_o...> about it raised some pretty major concerns about clarity, and I have to admit, I think if you're learning Python as a first language, the ?and, ?else, x?.attr, etc syntax is likely to be very confusing. For me personally, combining a new operator "?" with existing keywords like "and" or "else" just does not make any intuitive sense. I definitely see the value, though, in particular of None-severing, especially as a tool to explicitly specify which attr can be missing -- ie, disambiguating which attribute is missing in a foo.bar.baz lookup (the alternative to which is nested try: except AttributeError: blocks, which gets very messy very quickly). I'm on board with the idea, and I can absolutely imagine using it in my code, but I disagree on the spelling. A thought I had (perhaps more readable in a reddit comment <https://www.reddit.com/r/Python/comments/59d2he/pep_531_existence_checking_o...>) is to condense everything into a single "?" symbol, used for: + Coalescing binary operator: foo ? bar + Coalescing augmented assignment operator: foo ?= bar + Severing unary operator: ?foo *Pseudocode binary operator examples:*
foo_exists ? bar_never_evaluated foo_exists
foo_missing ? foo_exists foo_exists
foo_missing ? bar_missing foo_missing
*Pseudocode augmented examples:*
*Pseudocode unary examples:*
I personally think that's substantially more readable, but I suppose that's at least somewhat a matter of personal preference.

perhaps just having a utility function can get us some of the way there.. #may error r = a.b.x.z # will default to None r = a?.b?.x?.z r = get_null_aware(a, "b.x.z") # long but no new syntax, can be implemented today.

On Thu, Oct 27, 2016, at 11:27, Joonas Liik wrote:
If a.b can't or shouldn't be None, this should be a?.b.x.z I'm not certain how your utility function is supposed to differentiate this case, or handle subscripts or method calls.
r = get_null_aware(a, "b.x.z") # long but no new syntax, can be implemented today.

On Oct 27, 2016, at 06:27 PM, Joonas Liik wrote:
You could probably do this by extending operator.attrgetter() to take an optional 'coalesce' keyword. It wouldn't be super pretty, but it has the advantage of no magical new syntax. E.g. your example would be: from operator import attrgetter r = attrgetter('b.x.z', coalesce=True) That might be good enough for honestly how rare I think this use case is. (Similarly with itemgetter().) Cheers, -Barry

The problem with doing that is that it's ambiguous. There's no way of telling which attribute is allowed to coalesce. I think one of the best arguments for a coalescing operator in Python is that it allows you to be more explicit, without the hassle of nested try: except AttributeError blocks. You lose that with something like attrgetter('b.x.z', coalesce=True) -- it would behave identically, regardless of whether b, x, or z were missing, which is (oftentimes) not what you want. Nick Badger https://www.muterra.io https://www.nickbadger.com 2016-10-27 15:28 GMT-07:00 Barry Warsaw <barry@python.org>:

On Oct 27, 2016, at 07:37 PM, Nick Badger wrote:
The problem with doing that is that it's ambiguous. There's no way of telling which attribute is allowed to coalesce.
You could of course support exactly the same syntax being proposed as a language change, e.g. from operator import attrgetter r = attrgetter('b?.x?.z') and then you wouldn't even need the `coalesce` argument. Cheers, -Barry

On 28 October 2016 at 14:13, Barry Warsaw <barry@python.org> wrote:
The main drawback of this type of approach is that code checking tools will hardly ever support checking expressions inside the string like that. Also, you don't get proper syntax highlighting, code completion, etc. You can do anything you want by writing another programming language that is passed as string to a function, but that is not the same thing as having a proper syntax, is it? Just like type annotations with mypy: sure, you can add type annotations in comments, but it's not the same... -- Gustavo J. A. M. Carneiro Gambit Research "The universe is always one step beyond logic." -- Frank Herbert

On Oct 28, 2016, at 03:24 PM, Gustavo Carneiro wrote:
The bar for adding new language syntax is, and must be, high. Every new bit of syntax has a cost, so it has to be worth it. Guido deemed type annotations to be worth it and he may do the same for null coalescing operators. I don't personally think the need is so great or the use cases so common to incur that cost, but I'm just one opinion. The advantage of lower-cost approaches such as adopting the syntax in attrgetter() is that you piggyback on an existing API. Then you can use that as an experiment to see whether you really do solve enough problems in Python for a syntax change to be worth it. It's a lot like the ability to create properties and such before the syntactic sugar of decorators was added. I think that feature's pre-syntax popular and utility proved that the cost of adding syntax was worth it. Cheers, -Barry

On 10/15/16, Nick Coghlan <ncoghlan@gmail.com> wrote:
Please don't be too harsh to me for my next words! :) Just to trying to find something more cool (at least for young people :P ) I was thinking about emoticons... :( (means I am not happy because left is None) But parenthesis would be too confusing with "real" parenthesis. Example: (x :( (a + b)) # and editor would have problem to find corresponding pairs of parenthesis! So I am just trying to replace parenthesis with <> and playing with it * None-coalescing operator: x :< y * None-severing operator: x :> y * None-coalescing augmented assignment: x =< y * None-severing attribute access: x.>attr * None-severing subscript lookup: x[>expr] # ok this one seems a little weird to me too alternatively: x:>[expr] With this syntax/paradigm we could have also: * None-severing augmented assignment: x => y About spelling: we could say None-coalescing and None-severing operator in theory and sad and happy operator in real life. :) PL.

On Fri, Oct 14, 2016 at 11:36 PM, Guido van Rossum <guido@python.org> wrote:
I understand. You said the next phase should be to pick the best operator for each sub-proposal but I'm not sure how I can help with that. If there's something I can do, let me know and I'm happy to try to do it. In terms of "bunch of longer examples", what did you have in mind? I could take some popular library and rewrite a section of it with the proposed operators, but that would depend on the response to the previous paragraph.

Mark E. Haase writes:
I gather you think you have a deadlock here. The way to break it is to just do it. Pick a syntax and do the rewriting. My memory of some past instances is that many of the senior devs (especially Guido) will "see through the syntax" to evaluate the benefits of the proposal, even if they've said they don't particularly like the initially- proposed syntax. Unfortunately here the most plausible syntax is one that Guido has said he definitely doesn't like: using '?'. The alternatives are pretty horrible (a Haskell-like 'maybe' keyword, or the OPEN SQUARE character used by some logicians in modal logic -- the problem with the latter is that for many people it may not display at all with their font configurations, or it may turn into mojibake in email. OTOH, that case was an astral character -- after Guido announced his opposition to '?', the poster used PILE OF POO as the operator. OPEN SQUARE is in the basic multilingual plane, so probably is OK if the recipient can handle Unicode. '?' vs. '□': maybe that helps narrow the choice set?

On Sat, Oct 29, 2016 at 11:02:36AM +0900, Stephen J. Turnbull wrote:
I think you mean WHITE SQUARE? At least, I can not see any "OPEN SQUARE" code point in Unicode, and the character you use below □ is called WHITE SQUARE.
I cannot wait for the day that we can use non-ASCII operators. But I don't think that day has come: it is still too hard for many people (including me) to generate non-ASCII characters at the keyboard, and font support for some of the more useful ones are still inconsistent or lacking. For example, we don't have a good literal for empty sets. How about ∅? Sadly, in my mail client and in the Python REPR, it displays as a "missing glyph" open rectangle. And how would you type it? Ironically, WHITE SQUARE does display, but it took me a while to realise because at first I thought it too was the missing glyph character. And I still have no idea how to type it. Java, I believe, allows you to enter escape sequences in source code, not just in strings. So we could hypothetically allow one of: myobject\N{WHITE SQUARE}attribute myobject\u25a1attribute as a pure-ASCII way of getting myobject□attribute but really, who is going to do that? It is bad enough when strings contain escape sequences, but source code? So even though I *want* to use non-ASCI operators, I have to admit that I *can't* realistically use non-ASCII operators. Not yet. Wishing-that-somebody-can-prove-me-wrong-ly y'rs, -- Steve

On 29 October 2016 at 07:30, Steven D'Aprano <steve@pearwood.info> wrote:
So even though I *want* to use non-ASCI operators, I have to admit that I *can't* realistically use non-ASCII operators. Not yet.
Personally, I'm not even sure I want non-ASCII operators until non-ASCII characters are common, and used without effort, in natural language media such as email (on lists like this), source code comments, documentation, etc. For better or worse, it may be emoji that drive that change ;-) Paul

Steven d'Aprano writes:
You're right, I just used a common Japanese name for it. I even checked the table to make sure it was BMP but didn't notice the proper name which is written right there. Sorry for the confusion. Paul Moore writes:
The 3 billion computer users (and their ancestors) who don't live in the U.S. or Western Europe have been using non-ASCII, commonly, without effort, in natural language media on lists like this one for up to 5 decades now. In my own experience, XEmacs lists have explictly allowed Japanese and Russian since 1998, and used to see the occasional posts in German, French and Spanish, with no complaints of mojibake or objections that I can recall. And I have maintained XEmacs code containing Japanese identifiers, both variables and functions, since 1997. I understand why folks are reluctant, but face it, the technical issues were solved before half our users were born. It's purely a social problem now, and pretty much restricted to the U.S. at that.
For better or worse, it may be emoji that drive that change ;-)
I suspect that the 100 million or so Chinese, Japanese, Korean, and Indian programmers who have had systems that have no trouble whatsoever handling non-ASCII for as long they've used computers will drive that change.

On 29 October 2016 at 18:19, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
My apologies. You are of course absolutely right. I'm curious to know how easy it is for Chinese, Japanese, Korean and Indian programmers to use *ASCII* characters. I have no idea in practice whether the current basically entirely-ASCII nature of programming languages is as much a problem for them as I imagine Unicode characters would be for me. I really hope it isn't... Paul

On 29 October 2016 at 18:19, Stephen J. Turnbull <turnbull.stephen.fw at u.tsukuba.ac.jp> wrote:
The only way to do it http://ic.pics.livejournal.com/ibigdan/8161099/4947638/4947638_original.jpg Seriously, as a russian, I never had any problems with understanding that I should not go that far. I don't know of any axamples when using translit caused any personal problems in online conversations, unless it comes to quarrels and one tries to insult others for using translit. But russians are generally more minimalistically tuned than many other folks. As for returning non null, I suppose most readable way would be something like: non_null(a,b,c...) (sorry if I am missing the whole discussion topic, can easily happen with me since it is really mind blowing, why I would ever need it) Mikhail

Paul Moore writes:
tl;dr: A quick apology for the snark, and an attempt at FUD reduction. Using non-ASCII characters will involve some cost, but there are real benefits, and the fear and loathing often evoked by the prospect is unnecessary. I'm not ready to advocate introduction *right* now, but "never" isn't acceptable either. :-) On with the show: "Absolutely" is more than I deserve, as I was being a bit snarky. That said, Ed Yourdon wrote a book in 1990 or so with the self-promoting title of "Decline and Fall of the American Programmer"[1] in which he argued that for many kinds of software outsourcing to China, India, or Ireland got you faster, better, cheaper, and internationalized, with no tradeoffs. (The "and internationalized" is my hobby horse, it wasn't part of Yourdon's thesis.) He later recanted the extremist doomsaying, but a quick review of the fraction of H1B visas granted to Asian-origin programmers should convince you that USA/EUR/ANZ doesn't have a monopoly of good-to-great programming (probably never did, but that's a topic for a different thread). Also note that in Japan, without controlling for other factors, just the programming language used most frequently, Python programmers are the highest paid among developers in all languages with more than 1% of the sample (and yes, that includes COBOL!) To the extent that internationalization matters to a particular kind of programming, these programmers are better placed for those jobs, I think. And while in many cases "on site" has a big advantage (so you can't telecommute from Bangalore, you need that H1B which is available in rather restrictive number), more and more outsourcing does cross oceans so potential competition is immense. There is a benefit to increasing our internationalization in backward- incompatible ways. And that benefit is increasing both in magnitude and in the number of Python developers who will receive it.
Characters are zero problem for them. The East Asian national standards all include the ASCII repertoire, and some device (usually based on ISO 2022 coding extensions rather than UTF-8) for allowing ASCII to be one-byte, even if the "local" characters require two or more bytes. I forget if India's original national standard also included an ASCII subset, but they switched over to Unicode quite early[2], so UTF-8 does the trick for them. English (the language) is a much bigger issue. Most Indians, of course, have little trouble with the derived-from- English nature of much programming syntax and library identifiers, and the Asians all get enough training in both (very) basic English and rote memorization that handling English-derived syntax and library nomenclature is not a problem. However, reading and especially creating documentation can be expensive and inaccurate. At least in Japanese, "straightforward" translations are often poor, as nuances are lost. E.g., a literal Japanese translation from English requires many words to indicate the differences a simple "a" vs. "the" vs. "some" indicates in English. Mostly such nuances can be expressed economically by restructuring a whole paragraph, but translators rarely bother and often seem unaware of the issues. Many Japanese programmers' use of articles is literally chaotic: it's deterministic but appears random to all but the most careful analysis.[3]
as I imagine Unicode characters would be for me. I really hope it isn't...
I think your imagination is running away with you. While I understand how costly it is for those over the age of 12 to develop new habits (I'm 58, and painfully aware of how frequently I balk at learning anything new no matter how productivity-enhancing it is likely to be, and how much more slowly it becomes part of my repertoire), the number of new things you would need to learn would be few, and frequently enough used, at least in Python. It's hard enough to get Guido (and the other Masters of Pythonic Language Design) to sign on to new ASCII syntax; even if in principle non-ASCII were to be admitted, I suspect the barrier there would be even higher. Most of Unicode is irrelevant to everybody. Mathematicians use only a small fraction of the math notation available to them -- it's just that it's a different small fraction for each field. The East Asians need a big chunk (I would guess that educated Chinese and Japanese encounter about 10,000 characters in "daily life" over a lifetime, while those encountered at least once a week number about 3000), but those that need to be memorized are a small minority (less than 5%) of the already defined Unicode repertoire. For Western programmers, the mechanics are almost certainly there. Every personal computer should have at least one font containing all characters defined in the Basic Multilingual Plane, and most will have chunks of the astral planes (emoji, rare math symbols, country flags, ...). Even the Happy Hacker keyboard has enough mode keys (shift, control, ...) to allow defining "3-finger salutes" for commonly-used characters not on the keycaps -- in daily life if you don't need a input method now, you won't need one if Python decides to use WHITE SQUARE to represent an operation you frequently use -- just an extra "control key combo" like the editing control keys (eg, for copy, cut, paste, undo) that aren't marked on any keyboard I have. I'm *not* advocating *imposing* the necessary effort on anyone right now. I just want to reduce the FUD associated with the prospect that it *might* be imposed on *you*, so that you can evaluate the benefits in light of the real costs. They're not zero, but they're unlikely to ruin your whole day, every day, for months.[4] "Although sometimes never is better than *right* now" doesn't apply here. :-) Footnotes: [1] India is a multiscript country, so faces the same pressure for a single, internationally accepted character set as the whole world does, albeit at a lower level. [2] "The American Programmer" was the name of Yourdon's consultancy's newsletter to managers of software projects and software development organizations. [3] Of course the opposite is true when I write Japanese. In particular, there's a syntactic component called "particle" (the closest English equivalent is "preposition", but particles have much more general roles) that I'm sure my usage is equally chaotic from the point of view of a native speaker of Japanese -- even after working in the language for 25 years! N.B. I'm good enough at the language to have written grant proposals that were accepted in it -- and still my usage of particles is unreliable. [4] Well, if your role involves teaching other programmers, their pushback could be a long-lasting irritant. :-(

On 30 October 2016 at 07:00, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
My point wasn't so much about dealing with the character set of Unicode, as it was about physical entry of non-native text. For example, on my (UK) keyboard, all of the printed keycaps are basically used. And yet, I can't even enter accented letters from latin-1 with a standard keypress, much less extended Unicode. Of course it's possible to get those characters (either by specialised mappings in an editor, or by using an application like Character Map) but there's nothing guaranteed to work across all applications. That's a hardware and OS limitation - the hardware only has so many keys to use, and the OS (Windows, in my case) doesn't support global key mapping (at least not to my knowledge, in a user-friendly manner - I'm excluding writing my own keyboard driver :-)) My interest in East Asian experience is at least in part because the "normal" character sets, as I understand it, are big enough that it's impractical for a keyboard to include a plausible basic range of characters, so I'm curious as to what the physical process is for typing from a vocabulary of thousands of characters on a sanely-sized keyboard. In mentioning emoji, my main point was that "average computer users" are more and more likely to want to use emoji in general applications (emails, web applications, even documents) - and if a sufficiently general solution for that problem is found, it may provide a solution for the general character-entry case. (Also, I couldn't resist the irony of using a :-) smiley while referring to emoji...) But it may be that app-specific solutions (e.g., the smiley menu in Skype) are sufficient for that use case. Or the typical emoji user is likely to be using a tablet/phone rather than a keyboard, and mobile OSes have included an emoji menu in their on-screen keyboards. Coming back to a more mundane example, if I need to type a character like é in an email, I currently need to reach for Character Map and cut and paste it. The same is true if I have to type it into the console. That's a sufficiently annoying stumbling block that I'm inclined to avoid it - using clumsy workarounds like referring to "the OP" rather than using their name. I'd be fairly concerned about introducing non-ASCII syntax into Python while such stumbling blocks remain - the amount of code typed outside of an editor (interactive prompt, emails, web applications like Jupyter) mean that editor-based workarounds like custom mappings are only a partial solution. But maybe you are right, and it's just my age showing. The fate of APL probably isn't that relevant these days :-) (or ☺ if you prefer...) Paul

On Sun, Oct 30, 2016 at 11:22 PM, Paul Moore <p.f.moore@gmail.com> wrote:
Before Unicode emoji were prevalent, ASCII emoticons dominated, and it's not uncommon for multi-character sequences to be automatically transformed into their corresponding emoji. It isn't hard to set something up that does these kinds of transformations for other Unicode characters - use trigraphs for clarity, and type "/:0" to produce "∅". Or whatever's comfortable for you. Maybe rig it on Ctrl-Alt-0, if you prefer shift-key sequences. ChrisA

On 30 October 2016 at 12:31, Chris Angelico <rosuav@gmail.com> wrote:
It's certainly not difficult, in principle. I have (had, I lost it in an upgrade recently...) a little AutoHotkey program that interpreted Vim-style digraphs in any application that needed them. But my point was that we don't want to require people to write such custom utilities, just to be able to write Python code. Or is the feeling that it's acceptable to require that? Paul

On 30 October 2016 at 23:39, Paul Moore <p.f.moore@gmail.com> wrote:
Getting folks used to the idea that they need to use the correct kinds of quotes is already challenging :) However, the main issue is the one I mentioned in PEP 531 regarding the "THERE EXISTS" symbol: Python and other programming languages re-use "+", "-", "=" etc because a lot of folks are already familiar with them from learning basic arithmetic. Other symbols are used in Python because they were inherited from C, or were relatively straightforward puns on such previously inherited symbols. What this means is that there aren't likely to be many practical gains in using the "right" symbol for something, even when it's already defined in Unicode, as we expect the number of people learning that symbology *before* learning Python to be dramatically smaller than the proportion learning Python first and the formal mathematical symbols later (if they learn them at all). This means that instead of placing more stringent requirements on editing environments for Python source code in order to use non-ASCII input symbols, we're still far more likely to look to define a suitable keyword, or assign a relatively arbitrary meaning to an ASCII punctuation symbol (and that's assuming we accept that a proposal will see sufficient use to be worthy of new syntax in the first place, which is far from being a given). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

Hi all, For those of you not aware, the Julia Programming Language [1] does make extensive use of (mathematical) unicode symbols in its standard library, even document a method of input [2] (hint tab completion). They go even further by recognizing some characters (like \oplus) that parse as operators and have predefined precedences, but no implementations, leaving them available to the user. Regardless of my personal feeling about that, I have observed that this does not seem to hinder Julia developement. Many developers seem to like it a lot. Though my sampling is heavily biased toward developers with a strong math background. So it might be a case study to actually see how this affect an existing language both technically and community wide. Cheers, -- M [1] : julialang.org [2] : http://docs.julialang.org/en/release-0.5/manual/unicode-input/ [3] : http://docs.julialang.org/en/release-0.5/manual/variables/#allowed-variable-... On Sun, Oct 30, 2016 at 7:02 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:

Steven D'Aprano wrote:
I will just share my view on the whole problematic, trying to concentrate more on the actual code look. So I see it all as a chain of big steps, roughly: 1. One defines *the real code* or syntax, this means: One takes a pen and a paper (photoshop/paint bucket) and *defines* the syntax, this means one defines everything as it is, including pixel precise spaces between operators, punctuation and so on. 2. One develops an application (IDE) which enables you to automatically load code file and (at least) view it *exactly* as you have defined it. 3. Only after that one starts to think about ASCII/unicode/Hangul (forgive me Lord) or whatever someone has defined as something useful/standard.
So this actually would be a possible kind of "bridge" from the real code to what is shown up in arbitrary text editing application or a mailing client. In other words, you believe that in Unicode table you'll find something useful for code definition, but I personally would not even start relying on that, also because it is merely down-top problem solving. Mikhail

On Mon, Oct 31, 2016 at 12:02:54AM +1000, Nick Coghlan wrote:
Depends on the symbol. Most people do study maths in secondary school where they will be introduced to symbols beyond the ASCII + - * / etc, for instance set union and intersection ∪ ∩, although your point certainly applies to some of the more exotic (even for mathematicians) symbols in Unicode.
Indeed. But there's only so many ASCII punctuation marks, and digraphs and trigraphs can become tiresome. And once people have solved the keyboard entry issue, it is no harder to memorise the "correct" symbol than some arbitrary ASCII sequence.
I see that Perl is leading the way here, supporting a large number of Unicode symbols: https://docs.perl6.org/language/unicode_entry.html I must say that it is kinda cute that Perl6 does the right thing for x². -- Steve

Steven D'Aprano writes:
In what sense is that "support"? What I see on that page is a lot of advice for the kind of people who are already using non-ASCII in Python, as I have been doing since 2001 or so.
I must say that it is kinda cute that Perl6 does the right thing for x².
Uh, as far as I can tell from that page, Perl has absolutely nothing to do with that. You enter the Unicode code point as hex, and if the font supports, you get the character. What Paul is arguing is that entering any character, non-ASCII or ASCII, as a hex code point or as an Alt+digits sequence, is a non-starter for our audience. Much as I'd like to disagree, I can't.

On Mon, Oct 31, 2016 at 12:19 PM, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
Back when I used a single codepage (IBM OEM, now called 437) and 256 characters, it wasn't unreasonable to memorize the alt-codes for most of those characters. I could do all the single-line and double-line characters from memory (might take me a couple of tries to get the right corner), and if I needed to mix line types, I could just look those up. But with all of Unicode? Totally impractical. You can't expect people to use the hex codes. ChrisA

On Mon, Oct 31, 2016 at 10:19:58AM +0900, Stephen J. Turnbull wrote:
In the sense that Perl 6 not only allows Unicode identifiers (as Python has for many years) but also Unicode operators and symbols. For example, you can use either the Unicode character ⊂ \N{SUBSET OF} or the ASCII trigraph (<) for doing subset tests.
You missed the bit that Parl 6 interprets "x²" in code as the equivalent of x**2 (x squared). In other words, ² behaves as a unary postfix operator that squares its argument. Likewise for ³, etc. You can even combine them: x³³ would be the same as x**33. There's more here: https://docs.perl6.org/language/unicode_texas -- Steve

MRAB writes:
That's a strange thing to do. It's more usual to use a _subscript_ to indicate an index: a₃ vs a³
Oh, we economic theorists do that too. It's typically a double-indexed array of parameters, where both rows and columns can be meaningfully be treated as vectors. So a₃ is the vector of quantities of good 3 produced by all firms, while a³ is the vector of quantities of all goods produced by firm 3. Or in analysis of international or interregional trade, there's an index indicating which country exports which good to which importing country. Some people put the good index in the superscript and the two countries in the subscript, others the opposite. IIRC, mathematical physicist use both subscript and superscript in tensor notation, nuclear physicists use one for atomic number and the other for atomic weight (and thus would expect both subscript and superscript to be treated lexically as identifier components, not as expression components). The point is not that polynomials are not the most common use of superscript notation -- I don't care one way or the other. It's that there are many uses, important to those fields, that aren't polynomials.

On Mon, Oct 31, 2016 at 12:39 AM, Paul Moore <p.f.moore@gmail.com> wrote:
There's a chicken-and-egg problem. So long as most people don't have tools like that, a language that requires them is going to be very annoying - but so long as no major language uses such characters, there's no reason for developers to set up those kinds of tools. Possibly the best way is a gentle introduction of alternative syntaxes. Since Python currently has no "empty set display" syntax, that seems like a perfect starting point. You can always type "set()", but that involves an actual function call; using ∅ gives a small performance boost, eliminates the risk of shadowing, etc, etc. All minor points, but could be convenient enough. Also, if repr(set()) returns "∅", it'll be easy for anyone to get hold of the character for copy/paste. As of 2016, I think it's not acceptable to *require* this, but it may be time to start making use of it, retaining ASCII-only digraphs and trigraphs, the way C has alternative spelling for braces and so on. Then time passes, most people will be comfortable using the characters themselves, and the digraphs/trigraphs can be deprecated, with new syntax not being given any. Pipe dream? ChrisA

My vim configuration for a year or two has looked something like this (the screenshot doesn't show the empty set symbol, but that's part of my conceal configuration: http://gnosis.cx/bin/.vim/after/syntax/python.vim). On Sun, Oct 30, 2016 at 7:13 AM, Chris Angelico <rosuav@gmail.com> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On Sun, Oct 30, 2016 at 10:16:19AM -0700, David Mertz wrote:
Oh nice! By the way, anyone looking at this in a browser may find that the browser defaults to treating it as Latin-1, which gives you mojibake. Just tell you browser to treat it as Unicode. -- Steve

On 30 October 2016 at 14:43, <tritium-list@sdamon.com> wrote:
Just picking a nit, here, windows will happily let you do silly things like hook 14 keyboards up and let you map all of emoji to them. Sadly, this requires lua.
Off topic, I know, but how? I have a laptop with an external and an internal keyboard. Can I map the internal keyboard to different characters somehow? Paul

Paul Moore writes:
How do you type the pound sign and the Euro sign? Are they on the UK keyboard? Or are you not in the UK and don't need them?
And yet, I can't even enter accented letters from latin-1 with a standard keypress, much less extended Unicode.
I'm pretty sure you can, but since I've been Windows-free for 20 years (except for a short period when I was treasurer for an NPO, and only used it to access the accounting system), I can't tell you what it is. On the Mac, you press alt/option plus a graphic key. Most result in what somebody decided are common non-ASCII characters (German sharp S, Greek lowercase mu, Greek upper- and lowercase sigma), but several are dead keys, producing accented characters when combined with a base character: tilde, accents acute and grave, and so on. Surely Windows has a similar system (I don't mean Alt+digits). (But maybe not, I didn't notice one in my brief Googling.)
You're right about the size. Korean is special, because the 11,000- odd Hangul are phonetic and generated algorithmically from a set of about 70 phonetic partial glyphs, divided into three groups. The same keys do multiple duty when typed in phonetic order. Other systems use the shift key. For the 100,000 Han ideographs[1], there are a wide variety of methods for entry by key sequence, ranging from code point entry to context-dependent phonetic entry of entire sentences as they would be spoken. Then, of course, there's voice recognition, and handwriting recognition (both static from the image, and dynamic, taking account of the order of pen strokes). The more advanced input methods not only take account of grammar, but also learn the users' habits, remember recent conversions, and predict coming keystrokes based on current context, offering several conversions based on plausible continuations.
Not for the Asian languages. For them, "character entry" in the sense of character-by-character has long since been obsoleted by predictive sentence-level phonetic methods. But emoji are a perfect example for the present purpose, since they don't have standard pronunciations (although probably many will get them based on the Unicode standard names). On systems with high- enough resolution displays, a palette showing the glyphs is the obvious solution. But that's not pleasant if you type quickly and need those characters frequently. I don't think there's an alternative for emoji though, except for personalized shortcut maps. Math symbols are similar, I think.
You probably have Control, Windows, Menu, Alt, and maybe a "function" key. If you're lucky, one labelled AltGr for "Alternate Graphic" is the obvious suspect. Some combination of the above probably allows entry of accented Latin-1 characters, miscellaneous Latin-1 (eg, sharp S), and a few oddballs (Greek letters, ligatures like oe, the leminiscate usually read infinity).
That's a sufficiently annoying stumbling block
It very well could be, although my Windows Google-foo isn't great. But this is what I found. For WHITE SQUARE, the Mac doesn't have a keyboard equivalent, but there's a standard way to set up a set of shortcut keys[2]: http://stackoverflow.com/questions/3685146/how-do-you-do-the-therefore-%E2%8... And I think you can also use the "Input Preferences" screen in System Preferences to set up a few of them. For Windows, it seems that Alt+decimal character codes, or hex Unicode followed by Alt+x are the built-in ways to enter characters not on your keyboard. It's also possible to set up "Math Autocorrect" to automatically convert keysequences according to https://blogs.msdn.microsoft.com/murrays/2011/08/29/sans-serif-mathematical-... but that's hardly obvious (although maybe it is if you're Dutch?) I have to wonder why so many people stick with a system that seems to hate its users. :-( Footnotes: [1] I'm counting several thousand Taiwanese standard glyphs whose pronunciation and meaning is no longer known (they're culled from old manuscripts), as well as each of the 2 or 3 variants of several thousand characters given simplified glyphs by the Japanese and PRC standard bodies, because all have separate Unicode codepoints assigned. [2] Note: I had to Google this because I use Japanese input methods: when I want a square I type the Japanese word for "square" and then press "next conversion" until the square I want shows up. This also works for most Greek letters and math symbols. This doesn't bother me, because it's normal for typing Japanese (and I do mix Japanese and English enough that I know that it doesn't bug me when I need such a character in an otherwise all-English text). I suspect it would be inadequate for someone who doesn't also type a language requiring a complex input method.

On 30 October 2016 at 14:51, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
They are on the keyboard. The £ sign is shift-3, the € sign uses the AltGr key (which is woefully underused on the standard UK keyboard driver - accented letters *should* be available using it :-()
Believe me, I've tried. But I should point out that I *don't* count the "official" way (Alt plus typing the numeric code out on the numeric keypad) as a viable option: 1. It only works for the current codepage, I believe. 2. It gets intercepted by applications (I just tried it here, in the gmail webapp, and got dumped out of the site to a google search page, I've no idea why).
It doesn't, by default. Specialised programs can customise keypresses, but I'd hate to teach Python to newcomers if I needed something like that. (And by "newcomers" I'd include all of my work colleagues, who are far from computer illiterate...)
And it's application specific - noted in the article, "One way any character can be entered into Word or OneNote (but not into PowerPoint, sigh) is"
I have to wonder why so many people stick with a system that seems to hate its users. :-(
OT, but in my case, because it's very good at making a lot of the key things you need to do easy. It's immensely hostile in many ways, but typically if you're finding that to be the case, you're pretty clearly doing something that's not part of the "core target audience". Like console programs, Unicode outside a specific code page, etc. But if you are sticking to the norm, it's great. A question, though. On Linux, (pick your distribution, but ideally "it doesn't matter") how would I type é, √, ☺ ? Assume any answer that starts with "look up the numeric code" is unacceptable, as is anything that only works in a specific application. I'm willing to accept a need for a one-off configuration of some mapping table to get √, but accented letters and "common" characters like smileys should really be available by default. Assume a qwerty keyboard, something like UK or US layout (because it's the English speakers who need the most help remembering that the whole world isn't ASCII :-)) I doubt it's that much easier than it is on Windows. My ideal is that something like what I defined in the above paragraph *is* the norm, for all computer users. It's just plain silly that English speakers can't type café, or a German friend's correctly spelled name, without effort. Anyhow, this is way off topic now. Paul

Paul Moore writes:
OMG.
I don't either.
A question, though. On Linux, (pick your distribution, but ideally "it doesn't matter")
It does matter, at least last I checked. Different distros default to different keyboard configurations. And it definitely matters what language you configure as your primary -- accented letters and punctuation used in that language will use AltGr, while those that aren't may require a mode switch or a COMPOSE-ACCENT-BASE sequence.
Because this is X11/Unix, the answer is "it depends." (For math symbols and emoji, the common denominator default would surely be selection from a palette.) I suspect if there was a popular programming language that used a half-dozen non-ASCII characters, a slew of applets to configure those characters onto the keymap would arise quickly, and one of those that provided relative sane default mappings would become TOOWTDI. This is definitely possible, but at the moment, aside from language-specific mappings we already have, there's no obvious set of default characters that "everybody" needs. So a consistent, discoverable system for Unix won't happen until there's a bunch of non-ASCII everybody needs (and that can't be treated algorithmically like smart quotes), and no programming language will impose that until there's a consistent discoverable system of non-ASCII keymaps. :-(

Paul Moore writes:
How do you type the pound sign and the Euro sign? Are they on the UK keyboard? Or are you not in the UK and don't need them?
And yet, I can't even enter accented letters from latin-1 with a standard keypress, much less extended Unicode.
I'm pretty sure you can, but since I've been Windows-free for 20 years (except for a short period when I was treasurer for an NGO, and only used it to access the accounting system), I can't tell you what it is. On the Mac, you press alt/option plus a graphic key. Most result in what somebody decided are common non-ASCII characters (German sharp S, Greek lowercase mu, Greek upper- and lowercase sigma), but several are dead keys, producing accented characters when combined with a base character: tilde, accents acute and grave, and so on. Surely Windows has a similar system (I don't mean Alt+digits). (But maybe not, I didn't notice one in my brief Googling.)
You're right about the size. Korean is special, because the 11,000- odd Hangul are phonetic and generated algorithmically from a set of about 70 phonetic partial glyphs, divided into three groups. The same keys do multiple duty when typed in phonetic order. Other systems use the shift key. For the 100,000 Han ideographs[1], there are a wide variety of methods for entry by key sequence, ranging from code point entry to context-dependent phonetic entry of entire sentences as they would be spoken. Then, of course, there's voice recognition, and handwriting recognition (both static from the image, and dynamic, taking account of the order of pen strokes). The more advanced input methods not only take account of grammar, but also learn the users' habits, remember recent conversions, and predict coming keystrokes based on current context, offering several conversions based on plausible continuations.
Not for the Asian languages. For them, "character entry" in the sense of character-by-character has long since been obsoleted by predictive sentence-level phonetic methods. But emoji are a perfect example for the present purpose, since they don't have standard pronunciations (although probably many will get them based on the Unicode standard names). On systems with high- enough resolution displays, a palette showing the glyphs is the obvious solution. But that's not pleasant if you type quickly and need those characters frequently. I don't think there's an alternative for emoji though, except for personalized shortcut maps. Math symbols are similar, I think.
You probably have Control, Windows, Menu, Alt, and maybe a "function" key. If you're lucky, one labelled AltGr for "Alternate Graphic" is the obvious suspect. Some combination of the above probably allows entry of accented Latin-1 characters, miscellaneous Latin-1 (eg, sharp S), and a few oddballs (Greek letters, ligatures like oe, the leminiscate usually read infinity).
That's a sufficiently annoying stumbling block
It very well could be, although my Windows Google-foo isn't great. But this is what I found. For WHITE SQUARE, the Mac doesn't have a keyboard equivalent, but there's a standard way to set up a set of shortcut keys[2]: http://stackoverflow.com/questions/3685146/how-do-you-do-the-therefore-%E2%8... And I think you can also use the "Input Preferences" screen in System Preferences to set up a few of them. For Windows, it seems that Alt+decimal character codes, or hex Unicode followed by Alt+x are the built-in ways to enter characters not on your keyboard.. It's also possible to set up "Math Autocorrect" to automatically convert keysequences according to https://blogs.msdn.microsoft.com/murrays/2011/08/29/sans-serif-mathematical-... but that's hardly obvious (although maybe it is if you're Dutch?) I have to wonder why so many people stick with a system that obviously hates users. :-( Footnotes: [1] I'm counting several thousand Taiwanese standard glyphs whose pronunciation and meaning is no longer known (they're culled from old manuscripts), as well as each of the 2 or 3 variants of several thousand characters given simplified glyphs by the Japanese and PRC standard bodies, because all have separate Unicode codepoints assigned. [2] Note: I had to Google this because I use Japanese input methods: when I want a square I type the Japanese word for "square" and then press "next conversion" until the square I want shows up. This also works for most Greek letters and math symbols. This doesn't bother me, because it's normal for typing Japanese (and I do mix Japanese and English enough that I know that it doesn't bug me when I need such a character in an otherwise all-English text). I suspect it would be inadequate for someone who doesn't also type a language requiring a complex input method.

Stephen J. Turnbull wrote:
I don't feel deadlocked, but I think you're right about committing to a syntax. So I updated the PEP, summarized here: 1. Spelling a new operator as a keyword is difficult due to backward compatibility. It can be done (see PEP-308 and PEP-492) but requires extreme care. 2. A keyword operator is considered less ugly than punctuation, but it makes assignment shortcut syntax very ugly. Assume the coalesce operator is "foo", then the assignment shortcut would be "x foo= y". This is unacceptable. 3. If eliminate the possibility of a keyword and focus on punctuation, we find that most people think "??" — the syntax that exists in several other mainstream languages — is ugly and not Pythonic. 4. However, any other punctuation spelling will be at least as ugly and will not have the benefit of being familiar to programmers who have seen null coalescing in other languages. 5. Therefore, the most reasonable spelling is to borrow the same spelling that other languages use, e.g. "??", "?.", and "?[". I did go down the road of trying to create a new keyword, trying some mundane ideas ("foo else bar") and some more exotic ideas ("try foo then bar"), but I don't know if those syntaxes are even parseable, and as I worked through a bunch of examples, I realized that all of the keywords I was trying were very awkward in practical use, especially when combined with other expressions. Therefore, I have updated the PEP with the punctuation mentioned above, and at this point the PEP can't go any farther. If the best spelling for this new operator is unacceptable, then there's no getting around that. This PEP should be rejected.

On 31 October 2016 at 15:51, Mark E. Haase <mehaase@gmail.com> wrote:
While I agree that there's little point arguing over spelling here - if the ? spelling is unacceptable we should just reject - I'm not sure that's the only sticking point remaining here. I still find the short-circuiting behaviour of ?. (and ?[) to be pretty confusing - and the fact that there's a long paragraph describing the behaviour, with lots of examples of the form "if you think that this example works like so, then you're wrong, and it actually does the following", suggests to me that I'm not going to be the only one struggling. Hopefully, the problem is simply the way the behaviour is presented, and a reworking of the description would make it all crystal clear - but it feels to me that there's some inherent complexity here that's an actual issue with the proposal. Having said that, it appears that the proposed behaviour is the same as in C# (can you just come out and say "C#", rather than hinting with the phrase "other popular languages" - if we're stealing the behaviour as is from C#, let's say so, and if not, can you include examples from more than one language?) Assuming that's the case, then the fact that it's not causing confusion to C# programmers is a definite point in its favour. Paul

The PEP combines ideas from several different languages. For example: * Both Dart and C# have "x ?? y" and "x?.y". * Dart has "x ??= y" and C# does not. * C# has short circuit semantics for "?." and Dart does not. * PHP has "??" but does not have "?." * Etc. Wikipedia lists a lot of other languages[1], but I don't have enough personal experience with any of them to cite them in the PEP. This is why I use the phrase "other mainstream languages" multiple times. If you think the safe navigation operator isn't presented clearly, I am willing to improve it. Is there a particular example that you're struggling with? The simplest explanation is that it works the way you would want it too, e.g. in "foo?.bar.baz", we don't want semantics that could lead to looking up "baz" as an attribute of None. Therefore, if "foo?.bar" evaluates to None, then ".baz" is short circuited — that attribute is not looked up. [1] https://en.wikipedia.org/wiki/Null_coalescing_operator#SQL On Mon, Oct 31, 2016 at 12:33 PM, Paul Moore <p.f.moore@gmail.com> wrote:

I think we should try to improve our intutition about these operators. Many things that are intuitively clear still require long turgid paragraphs in reference documentation to state the behavior unambiguously -- but that doesn't stop us from intuitively grasping concepts like a+b (when does b.__radd__ get called?) or @classmethod. The main case to build intuition for is "?." -- once you get that the "?[...]" operator works just the same. So here's my attempt: *In a series of attribute accesses like "foo.bar.baz.bletch", putting a `?` before a specific dot inserts a None check for the expression to the left and skips everything to the right when the None check is true.* We still need to clarify what we mean by "expression to the left" and "everything to the right", but in most situations you will guess right without thinking about it. The expression to the left is easy -- it's determined by syntactic operator precedence, so that if we have "x = y + foo.bar?.baz.bletch", the expression to the left of the "?." is just "foo.bar". (But see below -- you won't actually see such usage much.) For "everything to the right" it would seem we have some freedom: e.g. if we have "foo.bar?.baz(bletch)" is the call included? The answer is yes -- the concept we're after here is named "trailer" in the Grammar file in the source code ( https://github.com/python/cpython/blob/master/Grammar/Grammar#L119), and "primary" in the reference manual ( https://docs.python.org/3/reference/expressions.html#primaries). This means all attribute references ("x.y"), index/slice operations ("x[...]"), and calls ("x(...)"). Note that in almost all cases the "?." operator will be used in an context where there is no other operator of lower precedence before or after it -- given the above meaning, it doesn't make a lot of sense to write "1 + x?.a" because "1 + None" is always an error (and ditto for "x?.a + 1"). However it still makes sense to assign such an expression to a variable or pass it as an argument to a function. So you can ignore the preceding four paragraphs: just remember the simplified rule (indented and in bold, depending on your email client) and let your intuition do the rest. Maybe it can even be simplified more: *The "?." operator splits the expression in two parts; the second part is skipped if the first part is None.* Eventually this *will* become intuitive. The various constraints are all naturally imposed by the grammar so you won't have to think about them consciously. --Guido On Mon, Oct 31, 2016 at 9:33 AM, Paul Moore <p.f.moore@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)

On 31 October 2016 at 17:16, Guido van Rossum <guido@python.org> wrote:
Thanks. Yes, I agree that details in a spec are never particularly obvious, and we need an intuition of what the operator does if it's to be successful. Mark - I couldn't offer a specific rewording, precisely because I found the whole thing confusing. But based on Guido's post, I'd say that the "intuitive" explanation of the proposed operators should be right at the top of the PEP, in the abstract - and should be repeated as the first statement in the specification section for each operator. The details can then follow, including all of the corner cases. But I'd be inclined there to word the corner cases as positive statements, rather than negative ones. Take for example, the case "d?.year.numerator + 1" - you say """ Note that the error in the second example is not on the attribute access numerator . In fact, that attribute access is never performed. The error occurs when adding None + 1 , because the None -aware attribute access does not short circuit + . """ which reads to me as presenting the misconception (that the error was from the access to numerator) before the correct explanation, and then explaining to the reader why they were confused if they thought that. I'd rather see it worded something along the lines of: """
d = None d?.year.numerator + 1
Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: unsupported operand type(s) for +: 'NoneType' and 'int' Note that the second example demonstrates that when ?. splits the enclosing expression into 2 parts, operators like + have a lower precedence, and so are not short circuited. So, we get a TypeError if d is None, because we're trying to add None to an integer (as the error states). """ There's no need to even *mention* the incorrect interpretation, it does nothing for people who'd misinterpreted the example in the first place, but for people who hadn't, it just suggests to them an alternative explanation they hadn't thought of - so confusing them where they weren't confused before. Does this clarify what I was struggling with in the way the PEP was worded? Paul

On 2016-10-31 17:16, Guido van Rossum wrote:
Would it help if we referred to them collectively as "suffixes"? Coincidentally, David Mertz's post includes a case where this feature would shorten the code. In normal Python form his code has: if x in stop_on or (end_if and end_if(x)): With this feature it could be: if x in stop_on or end_if?(x):

On Mon, Oct 31, 2016, at 13:16, Guido van Rossum wrote:
One thing that I think I touched on in an earlier iteration of this discussion but hasn't been revisited is: what's the AST going to look like? Right now, foo.bar.baz(bletch) is Call(Attribute(Attribute(Name('foo'), 'bar'), 'baz'), [Name('bletch')])), which is identical to (foo.bar).baz(bletch) or (foo.bar.baz)(bletch). These are treated, essentially, as postfix operators, where you can parenthesize any left part of the expression and leave its meaning [and its AST] unchanged. Is the AST going to be unchanged, leading to the conclusion that the short-circuiting in (foo?.bar).baz will "reach outside of" the parentheses, and relying on the fact that wanting to do that with None is a silly thing to do in almost all cases? Or is there going to be a new kind of AST that is sequential rather than recursive in how it represents trailer/primary expressions?

On Mon, Oct 31, 2016 at 05:44:07PM -0400, Random832 wrote:
I hope not. Even if it is "a silly thing to do in almost all cases", nevertheless it makes it hard to think about code if the ?. operator can reach outside of parentheses. If it can do that, just how far outside will it reach? Besides, "almost all" is not "all". For example: spam?.attr.__class__.__name__ (spam?.attr).__class__.__name__ I expect[1] that the first case would be equivalent to: None if spam is None else spam.attr.__class__.__name__ and the second case would be equivalent to: (None if spam is None else spam.attr).__class__.__name__ Concrete example: given spam = None, the first unparenthised version will return None, while the second parenthised version will return 'NoneType'. I don't know when I would ever want to actually do this in practice, but allowing the ?. operator to magically effect code outside of the parentheses definitely counts as "spooky action at a distance". Guido's rule of "everything to the right" is easy to reason about if "to the right" ends where the parenthised expression ends. [1] Actually I don't, or at least I didn't. I expected ?. to apply only to a single look-up. But Guido's description of the "everything to the right" rule seems like it will probably be more useful in practice and allow us to avoid writing long chains of spam?.eggs?.cheese?.tomato. So I'm changing my expectations. -- Steve

On Wed, Nov 2, 2016 at 11:09 AM, Steven D'Aprano <steve@pearwood.info> wrote:
We already expect "to the left" and "to the right" to end based on operator precedence rules. Parentheses are used to control operator precedence. It would surprise people *greatly* if they didn't bound the effect of the question mark. ChrisA

On 1 November 2016 at 01:51, Mark E. Haase <mehaase@gmail.com> wrote:
I do think it would be worth covering the symbol+keyword option discussed in PEP 531 (i.e. "?else" instead of "??", but keeping "?.", and "?[") Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 1 November 2016 at 10:11, Nick Coghlan <ncoghlan@gmail.com> wrote:
FWIW, I'm not keen on it. As a technical question, would it be treated in the syntax as a keyword, which happened to be made up of a punctuation character followed by letters, or as a ? symbol followed by a keyword? The difference would be in how "? else" was treated. If the space is not allowed, we have a unique situation in Python's grammar (where whitespace between a symbol and a keyword is explicitly disallowed rather than being optional). If it is allowed, I suspect a lot of people would prefer to write "? else" and aesthetically the two seem very different to me. Paul

On 1 November 2016 at 20:28, Paul Moore <p.f.moore@gmail.com> wrote:
Combined keyword, rather than two distinct tokens. If you don't do that, you end up creating ambiguities for other possible uses of "?" (like the "." and "?[]" suggestions) Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

But I also recall learning CoffeeScript via cargo-culting a large existing codebase and having not the foggiest ideas when it made sense to use ?. and when plain . was enough. So I think this feature is not very new-user-friendly and I still expect that in the end we'll have two rejected PEPs. But first we need to agree on what even the right definition of ?. is. It's been frighteningly difficult to explain this even on this list, even though I have a very clear view in my head, and PEP 505 also has the same semantics and explains well why those are the right semantics. So that's another point against this. It's syntactic sugar causing lots of cavities. On Wed, Nov 2, 2016 at 7:59 AM, Nick Coghlan <ncoghlan@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)

On 3 November 2016 at 01:46, Guido van Rossum <guido@python.org> wrote:
Yeah, and so far the protocol based alternative I'm working on hasn't been any less headache-inducing (Mark has been reviewing some early iterations and had to draw a diagram to try to follow the proposed control flow). I think I have a way to simplify that idea further though, and if that works out I should have something I'm willing to share with a wider audience. The gist is that rather than writing the bare: target = expr1 ?? expr2 ?? expr3 You'd instead write None-coalescing as: target = exists(expr1) ?? exists(expr2) ?? expr3 and None-propagating as: target = missing(expr1) ?? missing(expr2) ?? expr3 with ?? being a protocol-driven short-circuiting binary operator controlled by the left operand rather than defining any particular semantics of its own. The "obj?." and "obj?[]" would then be shorthand for particular uses of "missing(obj) ?? ..." that avoid duplicate evaluation of the left operand (as well as bypassing the overhead of actually creating a "missing" instance). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 02.11.2016 17:17, Nick Coghlan wrote:
I am sorry that I had to read this twice to get what you mean.
That could work if we accept that ?. and ?[] is only about the left hand-side. However, as the ? visually applies also to the attribute/index access on the right, which means it could be a more readable way of doing getattr(x, 'attr', None) or x[0] if len(x) > 0 else None Imagine you need to work with incomplete data and do some chaining with the new syntax. As soon as one single attribute isn't there, we hit an exception. The same for index. If the ?. and ?[] are introduced, I think applying the "non-existence" property also the right hand make sense here as well for a wide range of applications. Best, Sven

On 2 November 2016 at 19:34, MRAB <python@mrabarnett.plus.com> wrote:
Sorry for intruding into discussion and off-topic again, Such things like result = a > b ? x : y is IMHO a syntactical herecy. Such things disgust me from programming. Why on earth one cannot just wrap it in function if (a > b) { return x; } else { return y; } and write: c = nicefunc(a,b) Which will be WAY more readable and compact, if one wants to be compact for some reason. Mikhail

Even though I really don't want new null-coalescing operators, I really appreciate the ternary operator in Python (or in C). On Wed, Nov 2, 2016 at 12:38 PM, Mikhail V <mikhailwas@gmail.com> wrote:
The problem here is that the general form isn't ONLY to return 'x' or 'y' but the decide between arbitrary values. Hard-coding the variables into the function loses 90%+ of the point. So the general function would need a signature like: c = nicefunc(a, b, x, y) The problem here is that this call might be: c = nicefunc(a, b, run_for_hours(), has_side_effects()) We only want ONE of 'x' and 'y' to eagerly evaluate. In the C or Python ternary we get exactly that. Obviously, that also happens in your fully spelled out if/else block too, but that's multiline and needs to setup variables not just be used as an expression. -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On 2 November 2016 at 21:50, David Mertz <mertz@gnosis.cx> wrote:
[Apologies for off-topic one more time] So you say "that's multiline" just as if a multiner is something bad. All evil comes from the wish to write things more compactly. Also what to evaluate or not evaluate lies solely on the compiler/interpreter so that is not what the user must think of. Anyway, I am too far from doing chain attribute selections, but for the above example, a one-liner: if (a > b) : run_for_hours() else has_side_effects() Is this different from above? For me it is way more readable and no need to learn/memorise new operators. Also, *anything* of less then 3 characters as an operator is most likely to be initially a fail, simply because it is hard to *see* it in a mess of code, unlike kewords and well formatted multiliner. Or for example to return non-null of multiple operands: value = not_a_function_but_selector ( a, b, c, d ) Cannot this principle be exposed to attribute selector? As said I am too far from these problematics, still I'll try, though it may be total nonsense I am telling here: with seek_chain(foo.bar.bee.buzz) : name = "got it" I mean do you really need to stick something in the middle of chain, why not leave the whole thing there and let the "seek_chain" do the job (whatever this must be, I don't have an idea) Mikhail

On Wed, 2 Nov 2016 at 17:34 Mikhail V <mikhailwas@gmail.com> wrote:
A quick mailing list etiquette lesson: if you know you're going off-topic then please start a new thread with a new subject line (see the subject line for this new thread as an example). Since "off-topic" means "new thread of discussion", then please just start a new thread. You can't assume that people are reading email through an email client which does threading, meaning those of us not interested in the off-topic offshoot have to read your email, realize it's off-topic, and then get passed that email to get to the next email that is on-topic (and this is still a time sink even when your first line of your reply is "this is off-topic"). So please be mindful of when you stray off-topic, and when you do then start a new thread to make managing different discussions easier for everyone.

On Thu, Nov 03, 2016 at 02:17:14AM +1000, Nick Coghlan wrote:
Even if your protocol idea pans out and is a good idea, it doesn't solve the use-cases that PEP 505 is intended to solve. PEP 505 is specifically for the cases where None *is* special, where you *don't* want "None or something like None", but specifically None and nothing else. Your protocol idea is not an alternative to 505, it is independent of 505: it wants a second generalised concept of "None or something else" (distinct from falsey values), while 505 is specifically about "None and nothing else". -- Steve

: Disclaimer: I haven't followed all of this discussion, so some or all of the following may already have been expressed better (and perhaps refuted better still). On Wed, 2016-11-02 at 08:46 -0700, Guido van Rossum wrote:
I think the proposed semantics for ?. are confusing (and the operator itself dangerous, but more on that later). If I write something like obj.attr, the failure mode I care about is that obj has no attribute attr, rather than that obj is specifically None (or one of a defined group of somewhat Nonelike objects). Clearly, in such a circumstance, obj is not what I expected it to be, because I thought it was going to have an attribute attr, and it doesn't. Whether that's because it's None, some other Nonelike object, False, or some other type of object altogether may or may not be relevant, but is certainly secondary to the fact that there's no attr for me to work with. Most often I will want the default behaviour that an AttributeError is raised, but often it would be useful to say I'd like some kind of default value back with e.g. getattr(obj, 'attr', default), and sometimes it would be useful for that default value to propagate through a chain of attribute accesses without having to write a convoluted mess of nested getattr calls. In the latter case it usually does not matter to me what type obj is, except that if there *were* some syntax in Python that allowed me to propagate a default value through a chain of failed attribute accesses, it would be quite frustrating to find that I couldn't use it because obj happened not to be one of an anointed cohort of Nonelike objects. I'm sure it's obvious by now that I think the intuitive behaviour of any ?. operator is to suppress AttributeError on the right hand side, rather than check for nullity on the left. Maybe this reveals naïvety on my part regarding some inherent design or implementation difficulty with suppressing errors on the right rather than checking nullity on the left. However, I think "my" semantics are at least as likely to be assumed by a non-expert as the alternative proposed in this thread, and that's going to cause a lot of confusion. ... which leads to a second point: this is dangerous. I'm going to try to be tactful here, bearing in mind Nick Coghlan's well-argued imprecations against denigrating other languages, but: I recently spent a few months writing Ruby for a living. Ruby has a &. operator with similar semantics to those proposed, as well as (in ActiveSupport) an obj.try(:attr) method closer to the exception-catching model described above. It has both of those things, as far as I can tell, because of an IMO misguided preference in the Rails world for returning nil rather than raising exceptions, which tends to cause errors to materialize far from their source. When errors materialize far from their source, they are harder to debug, and it can be very tempting to use the likes of &. and try() as band- aids, rather than thoroughly investigate the causes of such errors. The end result of this tendency is that errors materialize further still from their source, and code gets progressively harder to debug over time. I worry that introducing such band-aids into Python will encourage similar behaviour, and for that reason I'm opposed to introducing an operator with either null-coalescing or exception-suppressing behaviour. Further, if a library encourages chained attribute accesses which may fail mid-chain, then I would rather that library take responsibility for the consequences of its design at the cost of some magic (e.g. by catching and handling AttributeError) than introduce confusing new syntax into the language. However, I do think an alternative is possible: extend getattr(). getattr(obj, ('chain', 'of', 'attributes'), default) This is more verbose than an operator, but I think its intent is clearer, and its verbosity makes it less likely to be misused. If extending getattr is unacceptable, a new dig() function (either as a builtin or part of the stdlib) with the same semantics would work just as well. If this has already been proposed here, I apologise (and would appreciate a pointer to arguments against it, if anyone can find them without too much effort). Incidentally, Chris Angelico's PEP 463 "Exception-catching expressions" could result in a clearer-still idiom: (obj.chain.of.attributes except AttributeError: default) ... but I suppose that ship has sailed. -[]z.

On Nov 02 2016, Zero Piraeus <schesis-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
That means that you do not need null coalescing operators. They're not intended for your use-case, and you are not the target audience. Criticizing the proposal on this basis is like critizing it for not helping with asynchronous I/O. Best, -Nikolaus -- GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F »Time flies like an arrow, fruit flies like a Banana.«

On 11/02/2016 12:32 PM, Nikolaus Rath wrote:
However, it's definitely a point for the confusiness of it all. I like the idea of null-(coalescing|negation|borrowing|what-have-you), but I also very much appreciate the keyword approach used in Python. I can already feel the headache coming on from trying to read and decipher the various ? accesses... -- ~Ethan~

Zero Piraeus writes:
That's what you care about because you're working with a data model where obj should not be None, and so you want an AttributeError to be raised when your assumption is violated. That's fine. But other people have other needs. PEP-505 includes a script that detects examples of null-coalescing and safe navigation syntax so that we can run it on real projects. Rather than debating if None should be used in this way, I prefer to measure how frequently None is already being used this way. I'm sure it's obvious by now that I think the intuitive behaviour of any
?. operator is to suppress AttributeError on the right hand side, rather than check for nullity on the left.
I can't emphasize strongly enough (and the same is said in PEP-505): this is not an error handling syntax. It does not suppress any exceptions. It is a syntax for handling common patterns with None. These are patterns that demonstrably exist in real Python code, including the standard library. Rewriting that code to use this new syntax would not change the safety, semantics, or error handling of that code.
However, I do think an alternative is possible: extend getattr().
How do your proposed built-in handle this case?
my_date?.replace(microsecond=0).isoformat()
Incidentally, Chris Angelico's PEP 463 "Exception-catching expressions"
That doesn't have the same semantics as obj?.chain.of.attributes. If you haven't read the PEP, all of these issues are covered in great detail.

Zero Piraeus writes:
If it's an error, you shouldn't be trying to do anything about it, just let the exception happen. The proposed .? syntax is designed for cases where it's *not* an error for the object to be missing the attribute, *and* the correct action in that situation is to skip whatever you would have done otherwise. What needs to be decided is whether such use cases are frequent enough to justify special syntax. -- Greg

I actually think that Zero's point here is quite valid... At some earlier point in the thread, I believe that Nick Coughlin was saying that we should be asking ourselves _why_ we want to do something like this and the result of that discussion was because there is pain when working with "pseudo-structured" responses from various APIs. This use-case resonates with me as I work with JSON responses quite frequently. The thing about "pseudo-structured" data is that there isn't an agreed upon way to represent it. While one API might send a field with a `null` value in some cases, another API might send a field with no data (after all, why bother putting it in the response if it's going to be `null`? You're just wasting bytes on the wire). As a concrete case where fields are truly missing -- Most of the Google APIs accept a "fields" parameter that allows you to pair down what is actually included in the response (If you don't believe me, feel free to play around with it in their API explorer -- https://developers.google.com/google-apps/calendar/v3/reference/calendars/ge... ). So, while this proposal is specifically about a "Null coalescing operator", my guess is that users will think of it as a "Get the field if it exists, else short-circuit and return `None`". And it might take a lot of education to try to get everyone aligned on the same page around what the "if it exists" actually means. At first, I was thinking that perhaps _adding_ an operator would help the community to standardize around "If the field is missing, then put a `None` in there -- But unfortunately, I think that the JSON API example demonstrates that a really big use-case for this operator is in working with APIs that are likely written in different languages whose communities frequently don't follow the same norms that python has embraced. I suppose that the suggestion would be to write: foo.get('bar')?['baz'] if the `foo` dict may or may not have ` bar` key? On Wed, Nov 2, 2016 at 3:22 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
-- [image: pattern-sig.png] Matt Gilson // SOFTWARE ENGINEER E: matt@getpattern.com // P: 603.892.7736 We’re looking for beta testers. Go here <https://www.getpattern.com/meetpattern> to sign up!

On Thu, Nov 03, 2016 at 11:22:45AM +1300, Greg Ewing wrote:
The proposed .? syntax is designed for cases where it's *not* an error for the object to be missing the attribute,
No it is not. That is absolutely not what the syntax means. I'm sorry to single you out Greg, but have you read the PEP? Or even the *title* of the PEP? Or even the subject line of this email thread??? Okay maybe the subject line of this thread isn't too clear (what's Null?). But the PEP is absolutely clear that this is specifically for dealing with None, not arbitrary missing attributes. If you write: obj = [] # oops I meant {} obj?.update(mapping) you will still get an AttributeError, because lists don't have an update method.
I think that the PEP does a good job of demonstrating that use-cases for these None-aware operators are common. -- Steve

On 11/2/2016 2:30 PM, Zero Piraeus wrote:
I agree with this understanding. The problem with None-coalescing is it doesn't consider where None came from. I suspect enumerating the source of None values will reveal the majority of them are a result of `getattr(obj, attr)` returning None (or obj.get(attr) returning None). If your code deals with annotated objects, rather than strictly typed objects, you will have many instances of attributes resolving to None. Making specific classes for each of the annotations, or combinations of annotations, is prohibitive, so you don't. Rather, you use a few general types and set some attributes to None to indicate a property is not-relevant-for-this-object. None checks are type checks. They are late type checks .

On Wed, Nov 02, 2016 at 10:22:56PM -0400, Kyle Lahnakoski wrote:
I don't see why you think that's a problem. What does it matter where None comes from? It seems to me that the important factor is what you intend to do with it, not where it came from. If I write this code: None if obj is None else obj.attr what difference does it make where None came from? The important thing is that I'm treating obj == None as a known condition. Now if you *don't* want to treat None specially, then of course this feature will be of no interest to you. This feature isn't aimed at you, it is aimed at the many people who do treat None specially. If you prefer: # obj is known to be a specific type, but attr may be None None if obj.attr is None else obj.attr.flange() That would be written obj.attr?.flange() in the proposed new syntax, and obj.attr will only be evaluated once. But note that this is quite different from the situation that Zero is talking about. In Zero's code, he prefers for obj.attr to not exist at all rather than setting it to None. So rather than this: None if obj.attr is None else obj.attr.flange() he might write something similar to this: None if not hasattr(obj, 'attr') else obj.attr.flange() (or at least *I* might write that, under the circumstances Zero refers to.) And here's the good bit: under this proposal, we might simplify it to this: getattr(obj, 'attr', None)?.flange() which handles three cases for the price of one: - obj.attr doesn't exist at all, in which case return None; - obj.attr exists and is None, in which case return None; - obj.attr exists and is not None, in which case call its flange() method. To me, having a compact and expressive syntax to write that is very attractive.
Have you read the PEP? If you do, you'll see that the author has already collected potential examples of use from the standard library: https://github.com/python/peps/blob/master/pep-0505/find-pep505.out Looking at those examples of code, I don't think it is likely that the majority (or even a large minority) are the result of getattr. But even if they are, what difference does it make?
I'm afraid I don't understand this.
None checks are type checks. They are late type checks .
Call it a type check if you like, but type check or value check or "rhubarb", whatever you want to call it, any time you test for None: if expression is None ... there's a reasonable chance that the None-aware operators may help you write better, more expressive code without falling into the trap of using `or` to handle None. Not in *all* cases, but I think often enough. -- Steve

On 3 November 2016 at 11:38, Steven D'Aprano <steve@pearwood.info> wrote:
It may have mattered, if a getattr(obj, attr, coalesce=true) function would provide the same effect with no syntax change. That approach was suggested earlier in the thread, and met with a lukewarm response, but if it had turned out to actually address the majority of use cases, that would be a point in its favour. It's a relatively minor point, but a valid one. Paul

On Wed, Nov 02, 2016 at 08:46:54AM -0700, Guido van Rossum wrote:
This is Python-Ideas and with respect to the many fine people on this list, the culture of the list does tend strongly towards over- generalisation which can lead to confusion: it may be hard to understand a proposal when multiple competing/alternative proposals are flying past all at once. That should not necessarily be taken as an argument against the feature itself: once details are locked down, and alternatives dismissed, the feature may not be so hard to understand, even for beginners. Here's my attempt at an explanation. spam?.eggs is simply syntactic sugar for: None if spam is None else spam.eggs It is not a generalised way of catching AttributeError. Nor is it a generalised way of detecting arbitrary empty or missing values. It is specific to None. "spam" can be an expression, not just a name, in which case it is only evaluated once: database.lookup[expensive+calculation]?.attribute only performs the expensive calculation and lookup once. It makes sense to allow ?. to bind "all the way to the right" so that: spam?.eggs.cheese.tomato.aardvark is sugar for: None if spam is None else spam.eggs.cheese.tomato.aardvark The ?[] operator extends this to item lookup: spam?[eggs] is sugar for: None if spam is None else spam[eggs] and the ?? "None-coalescing" operator extends this to general expressions: spam ?? eggs being sugar for: eggs if spam is None else spam Again, spam is only evaluated once. -- Steve

Thanks Steven, this is great! so -- when all this started, I think one of the use cases was to clean up this really common idiom: self.an_arg = the_default if an_arg is None else an_arg so would that be: self.an_arg = the_default ?? an_arg That would be nice. Though the fact that I'm still not sure if that's correct makes me think this is not so intuitive! -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Thu, Nov 3, 2016 at 12:00 PM, MRAB <python@mrabarnett.plus.com> wrote:
self.an_arg = the_default if an_arg is None else an_arg
No, ?? is a bit like 'or', except that only None is falsey, so it would be:
self.an_arg = an_arg ?? the_default
thanks! and actually, that reads much better to me. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Thu, Nov 03, 2016 at 12:35:07PM -0700, Chris Barker wrote:
That suggests a possible different colour for this operator: `?or`. (Apologies if that's already been suggested and rejected earlier.) The None-aware "safe navigation" operators ?. and ?[] will be used where Python already uses punctuation: spam.eggs # ordinary attribute lookup spam?.eggs # None-aware attribute lookup spam[eggs] # ordinary item lookup spam?[eggs] # None-aware item lookup which is simple enough to remember: just prefix your usual operator with a question mark to make it None-aware. But one of the disadvantages of ?? as an operator is that it replaces a keyword with completely unrelated punctuation: spam or default # default only if spam is any Falsey value spam ?? default # default only if spam is None Compared to: spam ?or default gives us the same rule: prefix the `or` operator with ? to make it None- aware. -- Steve

On Thu, Nov 3, 2016 at 4:06 PM, Steven D'Aprano <steve@pearwood.info> wrote:
I like it! -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On 3 November 2016 at 23:06, Steven D'Aprano <steve@pearwood.info> wrote:
Agree 100%. Prefixing an operator with ? meaning that it becomes None-aware is easy to remember. Just think of `?or` the same as `or`, but while `or` is truth-aware, `?or` would be None-aware. Similarly to other operators like . and []. How many times have I wanted to do something like `foo or bar`, but then again this is not always correct in case foo is an object that can be true or false, in which case it has to be expanded to `foo if foo is not None else bar`. With the proposal, we could write `foo ?or bar`, which is intuitively similar to the well known `foo or bar` expression and therefore easy to pick up.
-- Gustavo J. A. M. Carneiro Gambit Research "The universe is always one step beyond logic." -- Frank Herbert

On 10.09.2016 19:14, Random832 wrote:
But in the end, there were too many possibilities (operator?, nomad objects?, syntax expansion?, something else?), issues with all those "?" all over the place, hiding errors this way (which was the most serious one), and uncertainty about the overall benefit of this syntax compared to better designs like "how to not use None in the first place" didn't lead to a result so far. Let's see those can be resolved. Sven

On Sat, Sep 10, 2016 at 6:01 AM, Arek Bulski <arek.bulski@gmail.com> wrote:
You can use 'or' for this, as long as you're okay with other falsey values being treated the same way. In a lot of cases, this isn't a problem. However, even if this is implemented, it would be in an expression context, so 'raise' would never work. For that, I'd just use the explicit statement form. ChrisA

On Fri, Sep 9, 2016 at 3:01 PM, Arek Bulski <arek.bulski@gmail.com> wrote:
This was proposed almost exactly a year ago, start reading here: https://mail.python.org/pipermail/python-ideas/2015-September/036289.html -- Zach

This idea has come up before. While I can see the use of it, to me at least that use doesn't feel nearly common enough to warrant dedicated syntax. In many cases, it is a "truthy" value you are looking for rather than `is not None` specifically. That has a convenient spelling: expr or instead If it really is the actual None-ness you are curious about, you need the slightly longer: expr if expr is not None else instead Your example seems to want to fall back to a statement suite rather than a value. To do that, you'd have to put the suite inside a function such as: def Raise(err): raise err And use it something like: self.totalsizeof or Raise(SizeofError(...)) On Fri, Sep 9, 2016 at 1:01 PM, Arek Bulski <arek.bulski@gmail.com> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

I'd note you can also save 4 characters by writing: instead if expr is None else expr On Fri, Sep 9, 2016 at 1:10 PM, David Mertz <mertz@gnosis.cx> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On 2016-09-09 21:01, Arek Bulski wrote:
'raise' is a statement, so it can't appear in an expression. This has been discussed before, so you might want to read this thread first: Null coalescing operators https://mail.python.org/pipermail/python-ideas/2015-September/036289.html

On Fri, Sep 09, 2016 at 10:01:44PM +0200, Arek Bulski wrote:
As Zach and MRAB mention, this was discussed last year. If I recall correctly, the discussion fizzled out without a solid conclusion. I think there's a PEP -- if not, there should be. I would be interested in revisiting this idea, but 3.6 feature freeze is only a day or two away and I won't have time to discuss this before then. So let's please drop this discussion until the 3.6 beta is released. -- Steve

Steven D'Aprano writes:
The problem, as I recall, was that there was a conclusion that 'maybe_None if maybe_None is None else default' and the less precise 'maybe_falsy or default' are sufficient if you just want to provide a default if a value is "null". The proponents pointed out that other operators would benefit from null coalescing, and then discussion petered out because there was no remotely plausible suggestion except to use '?' to indicate null-coalescing versions. But Guido has historically resisted any use of '?' whatsoever in Python syntax, and there weren't any good alternatives or strong proponents of ?-based syntax. I forget if Guido was very sympathetic to null-coalescing operators, given somebody came up with a good syntax. There was also an issue of whether SQL NULLs and similar constructs "should" be spelled None in Python, and if not, how would null coalescence be defined. Steve

On Sat, Sep 10, 2016, at 12:48, Stephen J. Turnbull wrote:
I forget if Guido was very sympathetic to null-coalescing operators, given somebody came up with a good syntax.
As I remember the discussion, I thought he'd more or less conceded on the use of ? but there was disagreement on how to implement it that never got resolved. Concerns like, you can't have a?.b return None because then a?.b() isn't callable, unless you want to use a?.b?() for this case, or some people wanted to have "a?" [where a is None] return a magic object whose attribute/call/getitem would give no error, but that would have to keep returning itself and never actually return None for chained operators.

The way I recall it, we arrived at the perfect syntax (using ?) and semantics. The issue was purely strong hesitation about whether sprinkling ? all over your code is too ugly for Python, and in the end we couldn't get agreement on *that*. Another problem is PEP 505 -- it is full of discussion but its specification is unreadable due to the author's idea to defer the actual choice of operators and use a strange sequence of unicode characters instead. If someone wants to write a new, *short* PEP that defers to PEP 505 for motivation etc. and just writes up the spec for the syntax and semantics we'll have a better starting point. IMO the key syntax is simply one for accessing attributes returning None instead of raising AttributeError, so that e.g. `foo?.bar?.baz` is roughly equivalent to `foo.bar.baz if (foo is not None and foo.bar is not None) else None`, except evaluating foo and foo.bar only once. On Sat, Sep 10, 2016 at 10:14 AM, Random832 <random832@fastmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)

On 10 September 2016 at 18:26, Guido van Rossum <guido@python.org> wrote:
If we're not looking to use all the other null-coalescing variants (?=, ?(), ...) - which is something I'm pleased about, as I do think that scattering that many ?'s about is likely to lead to ugly code - then it would probably be fine to just use ? for this operation, so we'd have foo?bar?baz rather than needing foo?.bar?.baz. Paul

On Sat, Sep 10, 2016 at 11:09 AM, MRAB <python@mrabarnett.plus.com> wrote:
Indeed. And ?. is how this is spelled in some other lanuages (C# and Dart). I forgot one detail that's in PEP 505: e.g. `foo?.bar.baz()` should be implemented as `foo.bar.baz() if foo is not None else None`. IOW if foo is None, the entire trailing section `.bar.baz()` should be skipped. (But this is a property of `?.` as an alternative attribute access operator; it doesn't mean `?` is a postfix operator on `foo`.) Another issue already discussed in PEP 505 is a conflict with IPython (Jupyter Notebook), which uses ? and ?? as custom syntax to request help. But maybe it can be taught to only recognize those when they're the last character(s) on the line? -- --Guido van Rossum (python.org/~guido)

On Sat, Sep 10, 2016 at 4:56 PM, Guido van Rossum <guido@python.org> wrote:
I think this is already the case: In [1]: ?foo Object `foo` not found. In [2]: foo? Object `foo` not found. In [3]: foo?bar File "<ipython-input-3-c7de2a05ce6b>", line 1 foo?bar ^ SyntaxError: invalid syntax

On Sat, Sep 10, 2016, at 13:26, Guido van Rossum wrote:
I think that if there's "strong hesitation" about something being "too ugly" it can't really be described as "the perfect syntax". IIRC there were a couple alternatives being discussed that would have reduced the number of question marks to one [or one per object which might be None].

I find the '?.' syntax very ugly, much more so in the examples of chained attributes. A much better way to handle the use case is to wrap objects in a class that gives this "propagating None" behavior with plain attribute access. A nice implementation was presented in this thread. On Sep 10, 2016 3:16 PM, "Random832" <random832@fastmail.com> wrote:

https://github.com/kirbyfan64/_frozensafemockobjectimplementation In all seriousness, though, I really feel like that would be the ultimate bug magnet, since it'd be easy to forget to un-wrap the object afterwards. -- Ryan [ERROR]: Your autotools build scripts are 200 lines longer than your program. Something’s wrong. http://kirbyfan64.github.io/ On Sep 10, 2016 5:21 PM, "David Mertz" <mertz@gnosis.cx> wrote:

Actually, I guess the example I liked was from the year ago discussion. And it didn't do *exactly* what I think a wrapper should. What I'd want would be like this: class NoneCoalesce(object): "Standard operations on object for 'is not None'" def __init__(self, obj): self.obj = obj def __getattr__(self, name): try: return getattr(self.obj, name) except AttributeError: return NoneCoalesce(None) def __getitem__(self, item): try: return self.obj[item] except (TypeError, KeyError): return NoneCoalesce(None) def __call__(self, *args, **kwds): try: return self.obj(*args, **kwds) except TypeError: return NoneCoalesce(None) def __bool__(self): return self.obj is not None def __repr__(self): return "NoneCoalesce[%r]" % self.obj def __str__(self): return "NoneCoalesce[%r]" % self.obj def __len__(self): try: return len(self.obj) except TypeError: return 0 Then we might use it similar to this:
Nothing special about boltons' OrderedMultiDict here, just something I've been playing with that has some distinctive methods. The idea is that we can easily have both "regular" behavior and None coalescing just by wrapping any objects in a utility class... and WITHOUT adding ugly syntax. I might have missed some corners where we would want behavior wrapped, but those shouldn't be that hard to add in principle. On Sat, Sep 10, 2016 at 3:21 PM, David Mertz <mertz@gnosis.cx> wrote:
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.

On Sun, Sep 11, 2016 at 9:10 AM, Guido van Rossum <guido@python.org> wrote:
So you're offering `NoneCoalesce(x).bar` as less-ugly alternative to `x?.bar`... Color me unconvinced.
As a syntactic form? Not interested. But what if it's the underlying implementation? We have "yield from X" as a tidy syntax for roughly a page of equivalent code. We could have "x?.bar" as syntactic sugar for "NoneCoalesce(x).bar". ChrisA

On Sat, Sep 10, 2016 at 4:27 PM, Chris Angelico <rosuav@gmail.com> wrote:
PEP 505 has an option for a way to customize the coalescing operation (https://www.python.org/dev/peps/pep-0505/#generalized-coalescing). Though I think I'd rather not do that. But it just occurs to me that the implementation given by David Mertz is not what I'd expect: it seems that `NoneCoalesce([]).flup` would catch the AttributeError (there' no `[].flup`) and return NoneCoalesce(None), whereas I would expect `?.` to only return None when the LHS is None, not when some other not-None object doesn't have the requested attribute. (And the "pile of poo" operator in PEP 505 agrees with me.) -- --Guido van Rossum (python.org/~guido)

(Replying to multiple posts in this thread) Guido van Rossum:
Hi, I wrote PEP-505. I'm sorry that it's unreadable. The choice of emoji as operators was supposed to be a blatant joke. I'd be happy to submit a new version that is ASCII. Or make any other changes that would facilitate making a decision on the PEP. As I recall, the thread concluded with Guido writing, "I'll have to think about this," or something to that effect. I had hoped that the next step could be a survey where we could gauge opinions on the various possible spellings. I believe this was how PEP-308 was handled, and that was a very similar proposal to this one. Most of the discussion on list was really centered around the fact that nobody like the proposed ?? or .? spellings, and nobody could see around that fact to consider whether the feature itself was intrinsically valuable. (This is why the PEP doesn't commit to a syntax.) Also, as unfortunate side effect of a miscommunication, about 95% of the posts on this PEP were written _before_ I submitted a complete draft and so most of the conversation was arguing about a straw man. David Mertz:
The biggest problem with a wrapper in practice is that it has to be unwrapped before it can be passed to any other code that doesn't know how to handle it. E.g. if you want to JSON encode an object, you need to unwrap all of the NullCoalesce objects because the json module wouldn't know what to do with them. The process of wrapping and unwrapping makes the resulting code more verbose than any existing syntax.
There's a script in the PEP-505 repo that can you help you identify code that could be written with the proposed syntax. (It doesn't identify blocks that would not be affected, so this doesn't completely answer your question.) https://github.com/mehaase/pep-0505/blob/master/find-pep505.py The PEP also includes the results of running this script over the standard library. On Sat, Sep 10, 2016 at 1:26 PM, Guido van Rossum <guido@python.org> wrote:

I actually think the spelling is the main stumbling block. The intrinsic value of the behavior is clear, it's finding an acceptable spelling that hold back the proposal. I propose that the next phase of the process should be to pick the best operator for each sub-proposal. Then we can decide which of the sub-proposals we actually want in the language, based on a combination of how important the functionality is and how acceptable we find the spelling. --Guido On Thu, Oct 13, 2016 at 8:20 PM, Mark E. Haase <mehaase@gmail.com> wrote:
-- --Guido van Rossum (python.org/~guido)

For what it's worth, I like the C# syntax with question marks. It is probably more risky (breaks more code) to introduce a new keyword than a new symbol as operator. If we have to pick a symbol, it's less confusing if we pick something another language already uses. There is no shame in copying from other languages. Many of them copy ideas from Python as well ;-) Thanks. On 14 October 2016 at 17:10, Guido van Rossum <guido@python.org> wrote:
-- Gustavo J. A. M. Carneiro Gambit Research "The universe is always one step beyond logic." -- Frank Herbert

On 15 October 2016 at 13:36, Guido van Rossum <guido@python.org> wrote:
Having been previously somewhere between -1 and -0, I've been doing a lot more data mining and analysis work lately, which has been enough to shift me to at least +0 and potentially even higher when it comes to the utility of adding these operators (more on that below). = Pragmatic aspects = Regarding the spelling details, my current preferences are as follows: * None-coalescing operator: x ?or y * None-severing operator: x ?and y * None-coalescing augmented assignment: x ?= y * None-severing attribute access: x?.attr * None-severing subscript lookup: x?[expr] (The PEP currently only covers the "or?" and "and?" operator spelling suggestions, but the latter three suggestions are the same as those in the current PEP draft) My rationale for this preference is that it means that "?" is consistently a pseudo-operator that accepts an expression on the left and another binary operator (from a carefully restricted subset) on the right, and the combination is a new short-circuiting binary operation based on "LHS is not None". The last three operations can be defined in terms of the first two (with the usual benefit of avoiding repeated evaluation of the subexpression): * None-coalescing augmented assignment: x = x ?or y * None-severing attribute access: x ?and x.attr * None-severing subscript lookup: x ?and x[expr] The first two can then be defined in terms of equivalent if/else statements containing an "x is not None" clause: * None-coalescing operator: x if x is not None else y * None-severing operator: y if x is not None else x Importantly, the normal logical and/or can be expanded in terms of if/else in exactly the same way, only using "bool(x)" instead of "x is not None": * Logical or: x if x else y * Logical and: y if x else x = Language design philosophy aspects = Something I think is missing from the current PEP is a high level explanation of the *developer problem* that these operators solve - while the current PEP points to other languages as precedent, that just prompts the follow on question "Well, why did *they* add them, and does their rationale also apply to Python?". Even the current motivating examples don't really cover this, as they're quite tactical in nature ("Here is how this particular code is improved by the proposed change"), rather than explaining the high level user benefit ("What has changed in the surrounding technology environment that makes us think this is a user experience design problem worth changing the language definition to help address *now* even though Python has lived happily without these operators for 25+ years?") With conditional expressions, we had the clear driver that folks were insisting on using (and teaching!) the "and/or" hack as a workaround, and introducing bugs into their code as a result, whereas we don't have anything that clear-cut for this proposal (using "or" for None-coalescing doesn't seem to be anywhere near as popular as "and/or" used to be as an if/else equivalent). My point of view on that is that one of the biggest computing trends in recent years is the rise of "semi-structured data", where you're passing data around in either JSON-compatible data structures, or comparable structures mapped to instances and attributes, and all signs point to that being a permanent state change in the world of programming rather than merely being a passing fad. The world itself is fuzzy and ambiguous, and learning to work effectively with semi-structured data better reflects that ambiguity rather than forcing a false precision for the sake of code simplification. When you're working in that kind of context, encountering "None" is typically a shorthand for "This entire data subtree is missing, so don't try to do anything with it", but if it *isn't* None, you can safely assume that all the mandatory parts of that data segment will be present (no matter how deeply nested they are). To help explain that, it would be useful to mention not only the corresponding operators in other languages, but also the changes in data storage practices, like PostgreSQL's native support for JSON document storage and querying ( https://www.postgresql.org/docs/9.4/static/functions-json.html ) as well as the emergence/resurgence of hierarchical document storage techniques and new algorithms for working with them. However, it's also the case that where we *do* have a well understood and nicely constrained problem, it's still better to complain loudly when data is unexpectedly missing, rather than subjecting ourselves to the pain of having to deal with detecting problems with our data far away from where we introduced those problems. A *lot* of software still falls into that category, especially custom software written to meet the needs of one particular organisation. My current assumption is that those of us that now regularly need to deal with semi-structured data are thinking "Yes, these additions are obviously beneficial and improve Python's expressiveness, if we can find an acceptable spelling". Meanwhile, folks dealing primarily with entirely structured or entirely unstructured data are scratching their heads and asking "What's the big deal? How could it ever be worth introducing more line noise into the language just to make this kind of code easier to write?" Even the PEP's title is arguably a problem on that front - "None-aware operators" is a proposed *solution* to the problem of making semi-structured data easier to work with in Python, and hence reads like a solution searching for a problem to folks that don't regularly encounter these issues themselves. Framing the problem that way also provides a hint on how we could *document* these operations in the language reference in a readily comprehensible way: "Operators for working with semi-structured data" Regards, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 15.10.2016 08:10, Nick Coghlan wrote:
Definitely true. Stricter rules are similar to "fail early", "no errors should pass silently" and the like. This stance is conveyed by Python as long as I know it.
That's where I like to see a common middle ground between those two sides of the table. I need to work with both sides for years now. In my experience, it's best to avoid semi-structured data at all to keep the code simple. As we all know and as you described, the world isn't perfect and I can only agree. However, what served us best in recent years, is to keep the "semi-" out of the inner workings of our codebase. So, handling "semi-" at the system boundary proved to be a reliable way of not breaking everything and of keeping our devs sane. I am unsure how to implement such solution, whether via PEP8 or via the proposal's PEP. It somehow reminds me of the sans-IO idea where the core logic should be simple/linear code and the difficult/problematic issues are solved at the systems boundary. This said, let me put it differently by using an example. I can find None-aware operators very useful at the outermost function/methods of a process/library/class/module: class FanzyTool: def __init__(self, option1=None, option2=None, ...): # what happens when option6 and option7 are None # and it only matters when option 3 is not None # but when ... Internal function/methods/modules/classes and even processes/threads should have a clear, non-wishy-washy way of input and output (last but not least also to do unit-testing on relatively sane level). def _append_x(self, s): return s + 'x' # strawman operation Imagine, that s is something important to be passed around many times inside of "FanzyTool". The whole process usually makes no sense at all, when s is None. And having each internal method checking for None is getting messy fast. I hope we can also convey this issue properly when we find an appropriate syntax.
That's indeed an extraordinarily good title as it describes best what we intend it to be used for (common usage scenarios). +1 Regards, Sven

On Oct 15, 2016, at 04:10 PM, Nick Coghlan wrote:
I'm sympathetic to (some of) the goals of PEP 505, as these issues do occasionally annoy me. But I'm not entirely convinced they are common enough or annoying enough to warrant special syntax, and I *really* dislike the introduction of a ? operator for these purposes. I'm also concerned about adopting too much generality muddling up what I think should be a narrowly targeted improvement to readability. The other thing to note is that, while I often use ternary operators for this now, checking against None isn't always the sole conditional. E.g. self.chain = (chain if chain is None or IChain.providedBy(chain) else config.chains[chain]) That being said, null-aware member access (NAMA) would be pretty handy occasionally. I'm less sure about the other forms. For me, the biggest benefit of NAMA is the short-circuiting of chained attribute access. I don't like the operator syntax because I find it less readable (harder for the eye to pick out), and because it isn't a completely obvious operation. But also because I generally want to chase the attributes all-or-nothing. For example, foo.bar.baz.qux but only if all the intermediary attributes resolve to non-Nones. I don't want to have to write foo.?bar.?baz.?qux I tried playing around with new keywords such as 'when' and 'unless', which seem a little nice although not a perfect fit. thing = foo.bar.baz.qux unless None thing = unless None then foo.bar.baz.qux thing = when foo.bar.baz.qux thing = foo.bar.baz.qux when not None I do like the idea of a keyword more than an operator, and disagree that a new keyword can't be introduced until Python 4. That's why we have __future__! Anyway, that's my $0.02. I trust Guido to DTPT (do the Pythonic thing :), even if that means rejecting the PEP. Cheers, -Barry

Am Freitag, 14. Oktober 2016 23:11:48 UTC-7 schrieb Nick Coghlan:
This is, more or less, the syntax added in Nick's PEP 531 draft <https://www.python.org/dev/peps/pep-0531/>. The reddit discussion <https://www.reddit.com/r/Python/comments/59d2he/pep_531_existence_checking_o...> about it raised some pretty major concerns about clarity, and I have to admit, I think if you're learning Python as a first language, the ?and, ?else, x?.attr, etc syntax is likely to be very confusing. For me personally, combining a new operator "?" with existing keywords like "and" or "else" just does not make any intuitive sense. I definitely see the value, though, in particular of None-severing, especially as a tool to explicitly specify which attr can be missing -- ie, disambiguating which attribute is missing in a foo.bar.baz lookup (the alternative to which is nested try: except AttributeError: blocks, which gets very messy very quickly). I'm on board with the idea, and I can absolutely imagine using it in my code, but I disagree on the spelling. A thought I had (perhaps more readable in a reddit comment <https://www.reddit.com/r/Python/comments/59d2he/pep_531_existence_checking_o...>) is to condense everything into a single "?" symbol, used for: + Coalescing binary operator: foo ? bar + Coalescing augmented assignment operator: foo ?= bar + Severing unary operator: ?foo *Pseudocode binary operator examples:*
foo_exists ? bar_never_evaluated foo_exists
foo_missing ? foo_exists foo_exists
foo_missing ? bar_missing foo_missing
*Pseudocode augmented examples:*
*Pseudocode unary examples:*
I personally think that's substantially more readable, but I suppose that's at least somewhat a matter of personal preference.

perhaps just having a utility function can get us some of the way there.. #may error r = a.b.x.z # will default to None r = a?.b?.x?.z r = get_null_aware(a, "b.x.z") # long but no new syntax, can be implemented today.

On Thu, Oct 27, 2016, at 11:27, Joonas Liik wrote:
If a.b can't or shouldn't be None, this should be a?.b.x.z I'm not certain how your utility function is supposed to differentiate this case, or handle subscripts or method calls.
r = get_null_aware(a, "b.x.z") # long but no new syntax, can be implemented today.

On Oct 27, 2016, at 06:27 PM, Joonas Liik wrote:
You could probably do this by extending operator.attrgetter() to take an optional 'coalesce' keyword. It wouldn't be super pretty, but it has the advantage of no magical new syntax. E.g. your example would be: from operator import attrgetter r = attrgetter('b.x.z', coalesce=True) That might be good enough for honestly how rare I think this use case is. (Similarly with itemgetter().) Cheers, -Barry

The problem with doing that is that it's ambiguous. There's no way of telling which attribute is allowed to coalesce. I think one of the best arguments for a coalescing operator in Python is that it allows you to be more explicit, without the hassle of nested try: except AttributeError blocks. You lose that with something like attrgetter('b.x.z', coalesce=True) -- it would behave identically, regardless of whether b, x, or z were missing, which is (oftentimes) not what you want. Nick Badger https://www.muterra.io https://www.nickbadger.com 2016-10-27 15:28 GMT-07:00 Barry Warsaw <barry@python.org>:

On Oct 27, 2016, at 07:37 PM, Nick Badger wrote:
The problem with doing that is that it's ambiguous. There's no way of telling which attribute is allowed to coalesce.
You could of course support exactly the same syntax being proposed as a language change, e.g. from operator import attrgetter r = attrgetter('b?.x?.z') and then you wouldn't even need the `coalesce` argument. Cheers, -Barry

On 28 October 2016 at 14:13, Barry Warsaw <barry@python.org> wrote:
The main drawback of this type of approach is that code checking tools will hardly ever support checking expressions inside the string like that. Also, you don't get proper syntax highlighting, code completion, etc. You can do anything you want by writing another programming language that is passed as string to a function, but that is not the same thing as having a proper syntax, is it? Just like type annotations with mypy: sure, you can add type annotations in comments, but it's not the same... -- Gustavo J. A. M. Carneiro Gambit Research "The universe is always one step beyond logic." -- Frank Herbert

On Oct 28, 2016, at 03:24 PM, Gustavo Carneiro wrote:
The bar for adding new language syntax is, and must be, high. Every new bit of syntax has a cost, so it has to be worth it. Guido deemed type annotations to be worth it and he may do the same for null coalescing operators. I don't personally think the need is so great or the use cases so common to incur that cost, but I'm just one opinion. The advantage of lower-cost approaches such as adopting the syntax in attrgetter() is that you piggyback on an existing API. Then you can use that as an experiment to see whether you really do solve enough problems in Python for a syntax change to be worth it. It's a lot like the ability to create properties and such before the syntactic sugar of decorators was added. I think that feature's pre-syntax popular and utility proved that the cost of adding syntax was worth it. Cheers, -Barry

On 10/15/16, Nick Coghlan <ncoghlan@gmail.com> wrote:
Please don't be too harsh to me for my next words! :) Just to trying to find something more cool (at least for young people :P ) I was thinking about emoticons... :( (means I am not happy because left is None) But parenthesis would be too confusing with "real" parenthesis. Example: (x :( (a + b)) # and editor would have problem to find corresponding pairs of parenthesis! So I am just trying to replace parenthesis with <> and playing with it * None-coalescing operator: x :< y * None-severing operator: x :> y * None-coalescing augmented assignment: x =< y * None-severing attribute access: x.>attr * None-severing subscript lookup: x[>expr] # ok this one seems a little weird to me too alternatively: x:>[expr] With this syntax/paradigm we could have also: * None-severing augmented assignment: x => y About spelling: we could say None-coalescing and None-severing operator in theory and sad and happy operator in real life. :) PL.

On Fri, Oct 14, 2016 at 11:36 PM, Guido van Rossum <guido@python.org> wrote:
I understand. You said the next phase should be to pick the best operator for each sub-proposal but I'm not sure how I can help with that. If there's something I can do, let me know and I'm happy to try to do it. In terms of "bunch of longer examples", what did you have in mind? I could take some popular library and rewrite a section of it with the proposed operators, but that would depend on the response to the previous paragraph.

Mark E. Haase writes:
I gather you think you have a deadlock here. The way to break it is to just do it. Pick a syntax and do the rewriting. My memory of some past instances is that many of the senior devs (especially Guido) will "see through the syntax" to evaluate the benefits of the proposal, even if they've said they don't particularly like the initially- proposed syntax. Unfortunately here the most plausible syntax is one that Guido has said he definitely doesn't like: using '?'. The alternatives are pretty horrible (a Haskell-like 'maybe' keyword, or the OPEN SQUARE character used by some logicians in modal logic -- the problem with the latter is that for many people it may not display at all with their font configurations, or it may turn into mojibake in email. OTOH, that case was an astral character -- after Guido announced his opposition to '?', the poster used PILE OF POO as the operator. OPEN SQUARE is in the basic multilingual plane, so probably is OK if the recipient can handle Unicode. '?' vs. '□': maybe that helps narrow the choice set?

On Sat, Oct 29, 2016 at 11:02:36AM +0900, Stephen J. Turnbull wrote:
I think you mean WHITE SQUARE? At least, I can not see any "OPEN SQUARE" code point in Unicode, and the character you use below □ is called WHITE SQUARE.
I cannot wait for the day that we can use non-ASCII operators. But I don't think that day has come: it is still too hard for many people (including me) to generate non-ASCII characters at the keyboard, and font support for some of the more useful ones are still inconsistent or lacking. For example, we don't have a good literal for empty sets. How about ∅? Sadly, in my mail client and in the Python REPR, it displays as a "missing glyph" open rectangle. And how would you type it? Ironically, WHITE SQUARE does display, but it took me a while to realise because at first I thought it too was the missing glyph character. And I still have no idea how to type it. Java, I believe, allows you to enter escape sequences in source code, not just in strings. So we could hypothetically allow one of: myobject\N{WHITE SQUARE}attribute myobject\u25a1attribute as a pure-ASCII way of getting myobject□attribute but really, who is going to do that? It is bad enough when strings contain escape sequences, but source code? So even though I *want* to use non-ASCI operators, I have to admit that I *can't* realistically use non-ASCII operators. Not yet. Wishing-that-somebody-can-prove-me-wrong-ly y'rs, -- Steve

On 29 October 2016 at 07:30, Steven D'Aprano <steve@pearwood.info> wrote:
So even though I *want* to use non-ASCI operators, I have to admit that I *can't* realistically use non-ASCII operators. Not yet.
Personally, I'm not even sure I want non-ASCII operators until non-ASCII characters are common, and used without effort, in natural language media such as email (on lists like this), source code comments, documentation, etc. For better or worse, it may be emoji that drive that change ;-) Paul

Steven d'Aprano writes:
You're right, I just used a common Japanese name for it. I even checked the table to make sure it was BMP but didn't notice the proper name which is written right there. Sorry for the confusion. Paul Moore writes:
The 3 billion computer users (and their ancestors) who don't live in the U.S. or Western Europe have been using non-ASCII, commonly, without effort, in natural language media on lists like this one for up to 5 decades now. In my own experience, XEmacs lists have explictly allowed Japanese and Russian since 1998, and used to see the occasional posts in German, French and Spanish, with no complaints of mojibake or objections that I can recall. And I have maintained XEmacs code containing Japanese identifiers, both variables and functions, since 1997. I understand why folks are reluctant, but face it, the technical issues were solved before half our users were born. It's purely a social problem now, and pretty much restricted to the U.S. at that.
For better or worse, it may be emoji that drive that change ;-)
I suspect that the 100 million or so Chinese, Japanese, Korean, and Indian programmers who have had systems that have no trouble whatsoever handling non-ASCII for as long they've used computers will drive that change.

On 29 October 2016 at 18:19, Stephen J. Turnbull <turnbull.stephen.fw@u.tsukuba.ac.jp> wrote:
My apologies. You are of course absolutely right. I'm curious to know how easy it is for Chinese, Japanese, Korean and Indian programmers to use *ASCII* characters. I have no idea in practice whether the current basically entirely-ASCII nature of programming languages is as much a problem for them as I imagine Unicode characters would be for me. I really hope it isn't... Paul

On 29 October 2016 at 18:19, Stephen J. Turnbull <turnbull.stephen.fw at u.tsukuba.ac.jp> wrote:
The only way to do it http://ic.pics.livejournal.com/ibigdan/8161099/4947638/4947638_original.jpg Seriously, as a russian, I never had any problems with understanding that I should not go that far. I don't know of any axamples when using translit caused any personal problems in online conversations, unless it comes to quarrels and one tries to insult others for using translit. But russians are generally more minimalistically tuned than many other folks. As for returning non null, I suppose most readable way would be something like: non_null(a,b,c...) (sorry if I am missing the whole discussion topic, can easily happen with me since it is really mind blowing, why I would ever need it) Mikhail

Paul Moore writes:
tl;dr: A quick apology for the snark, and an attempt at FUD reduction. Using non-ASCII characters will involve some cost, but there are real benefits, and the fear and loathing often evoked by the prospect is unnecessary. I'm not ready to advocate introduction *right* now, but "never" isn't acceptable either. :-) On with the show: "Absolutely" is more than I deserve, as I was being a bit snarky. That said, Ed Yourdon wrote a book in 1990 or so with the self-promoting title of "Decline and Fall of the American Programmer"[1] in which he argued that for many kinds of software outsourcing to China, India, or Ireland got you faster, better, cheaper, and internationalized, with no tradeoffs. (The "and internationalized" is my hobby horse, it wasn't part of Yourdon's thesis.) He later recanted the extremist doomsaying, but a quick review of the fraction of H1B visas granted to Asian-origin programmers should convince you that USA/EUR/ANZ doesn't have a monopoly of good-to-great programming (probably never did, but that's a topic for a different thread). Also note that in Japan, without controlling for other factors, just the programming language used most frequently, Python programmers are the highest paid among developers in all languages with more than 1% of the sample (and yes, that includes COBOL!) To the extent that internationalization matters to a particular kind of programming, these programmers are better placed for those jobs, I think. And while in many cases "on site" has a big advantage (so you can't telecommute from Bangalore, you need that H1B which is available in rather restrictive number), more and more outsourcing does cross oceans so potential competition is immense. There is a benefit to increasing our internationalization in backward- incompatible ways. And that benefit is increasing both in magnitude and in the number of Python developers who will receive it.
Characters are zero problem for them. The East Asian national standards all include the ASCII repertoire, and some device (usually based on ISO 2022 coding extensions rather than UTF-8) for allowing ASCII to be one-byte, even if the "local" characters require two or more bytes. I forget if India's original national standard also included an ASCII subset, but they switched over to Unicode quite early[2], so UTF-8 does the trick for them. English (the language) is a much bigger issue. Most Indians, of course, have little trouble with the derived-from- English nature of much programming syntax and library identifiers, and the Asians all get enough training in both (very) basic English and rote memorization that handling English-derived syntax and library nomenclature is not a problem. However, reading and especially creating documentation can be expensive and inaccurate. At least in Japanese, "straightforward" translations are often poor, as nuances are lost. E.g., a literal Japanese translation from English requires many words to indicate the differences a simple "a" vs. "the" vs. "some" indicates in English. Mostly such nuances can be expressed economically by restructuring a whole paragraph, but translators rarely bother and often seem unaware of the issues. Many Japanese programmers' use of articles is literally chaotic: it's deterministic but appears random to all but the most careful analysis.[3]
as I imagine Unicode characters would be for me. I really hope it isn't...
I think your imagination is running away with you. While I understand how costly it is for those over the age of 12 to develop new habits (I'm 58, and painfully aware of how frequently I balk at learning anything new no matter how productivity-enhancing it is likely to be, and how much more slowly it becomes part of my repertoire), the number of new things you would need to learn would be few, and frequently enough used, at least in Python. It's hard enough to get Guido (and the other Masters of Pythonic Language Design) to sign on to new ASCII syntax; even if in principle non-ASCII were to be admitted, I suspect the barrier there would be even higher. Most of Unicode is irrelevant to everybody. Mathematicians use only a small fraction of the math notation available to them -- it's just that it's a different small fraction for each field. The East Asians need a big chunk (I would guess that educated Chinese and Japanese encounter about 10,000 characters in "daily life" over a lifetime, while those encountered at least once a week number about 3000), but those that need to be memorized are a small minority (less than 5%) of the already defined Unicode repertoire. For Western programmers, the mechanics are almost certainly there. Every personal computer should have at least one font containing all characters defined in the Basic Multilingual Plane, and most will have chunks of the astral planes (emoji, rare math symbols, country flags, ...). Even the Happy Hacker keyboard has enough mode keys (shift, control, ...) to allow defining "3-finger salutes" for commonly-used characters not on the keycaps -- in daily life if you don't need a input method now, you won't need one if Python decides to use WHITE SQUARE to represent an operation you frequently use -- just an extra "control key combo" like the editing control keys (eg, for copy, cut, paste, undo) that aren't marked on any keyboard I have. I'm *not* advocating *imposing* the necessary effort on anyone right now. I just want to reduce the FUD associated with the prospect that it *might* be imposed on *you*, so that you can evaluate the benefits in light of the real costs. They're not zero, but they're unlikely to ruin your whole day, every day, for months.[4] "Although sometimes never is better than *right* now" doesn't apply here. :-) Footnotes: [1] India is a multiscript country, so faces the same pressure for a single, internationally accepted character set as the whole world does, albeit at a lower level. [2] "The American Programmer" was the name of Yourdon's consultancy's newsletter to managers of software projects and software development organizations. [3] Of course the opposite is true when I write Japanese. In particular, there's a syntactic component called "particle" (the closest English equivalent is "preposition", but particles have much more general roles) that I'm sure my usage is equally chaotic from the point of view of a native speaker of Japanese -- even after working in the language for 25 years! N.B. I'm good enough at the language to have written grant proposals that were accepted in it -- and still my usage of particles is unreliable. [4] Well, if your role involves teaching other programmers, their pushback could be a long-lasting irritant. :-(
participants (36)
-
Alexander Belopolsky
-
Alexandre Brault
-
Arek Bulski
-
Barry Warsaw
-
Brett Cannon
-
Chris Angelico
-
Chris Barker
-
David Mertz
-
Eric Snow
-
Ethan Furman
-
Greg Ewing
-
Guido van Rossum
-
Guido van Rossum
-
Gustavo Carneiro
-
Ivan Levkivskyi
-
Joonas Liik
-
Kyle Lahnakoski
-
Mark E. Haase
-
Matt Gilson
-
Matthias Bussonnier
-
Mikhail V
-
MRAB
-
Nick Badger
-
Nick Coghlan
-
Nikolaus Rath
-
Paul Moore
-
Pavol Lisy
-
Ralph Broenink
-
Random832
-
Ryan Gonzalez
-
Stephen J. Turnbull
-
Steven D'Aprano
-
Sven R. Kunze
-
tritium-list@sdamon.com
-
Zachary Ware
-
Zero Piraeus