Based on some emails I read in the " unpacking generalisations for list
comprehension", I feel like I need to address this entire list about its
general behaviour.
If you don't follow me on Twitter you may not be aware that I am taking the
entire month of October off from volunteering any personal time on Python
for my personal well-being (this reply is being done on work time for
instance). This stems from my wife pointing out that I had been rather
stressed in July and August outside of work in relation to my Python
volunteering (having your weekends ruined is never fun). That stress
stemmed primarily from two rather bad interactions I had to contend with on
the issue track in July and August ... and this mailing list.
When I have talked to people about this mailing list it's often referred to
by others as the "wild west" of Python development discussions (if you're
not familiar with US culture, that turn of phrase basically means "anything
goes"). To me that is not a compliment. When I created this list with Titus
the goal was to provide a safe place where people could bring up ideas for
Python where people could quickly provide basic feedback so people could
know whether there was any chance that python-dev would consider the
proposal. This was meant to be a win for proposers by not feeling like they
were wasting python-dev's time and a win for python-dev by keeping that
list focused on the development of Python and not fielding every idea that
people want to propose.
And while this list has definitely helped with the cognitive load on
python-dev, it has not always provided a safe place for people to express
ideas. I have seen people completely dismiss people's expertise and
opinion. There has been name calling and yelling at people (which is always
unnecessary). There have been threads that have completely derailed itself
and gone entirely off-topic. IOW I would not hold this mailing list up as
an example of the general discourse that I experience elsewhere within the
community.
Now I realize that we are all human beings coming from different cultural
backgrounds and lives. We all have bad days and may not take the time to
stop and think about what we are typing before sending it, leading to
emails that are worded in a way that can be hurtful to others. It's also
easy to forget that various cultures views things differently and so that
can lead to people "reading between the lines" a lot and picking up things
that were never intended. There are 1,031 people on this mailing list from
around the world and it's easy to forget that e.g. Canadian humour may not
translate well to Ukrainian culture (or something). What this means is it's
okay to *nicely* say that something bothered you, but also try to give
people the benefit of the doubt as you don't know what their day had been
like before they wrote that email (I personally don't like the "just mute
the thread" approach to dealing with bad actors when the muting is silent
as that doesn't help new people who join this mailing list and the first
email they see is someone being rude that everyone else didn't see because
they muted the thread days ago).
As for the off-topic threads, please remember there are 1,031 people on
this mailing list (this doesn't count people reading through gmane or
Google Groups). Being extremely generous and assuming every person on this
list only spends 10 seconds deciding if they care about your email, that's
still nearly 3 hours of cumulative time spent on your email. So please be
cognisant when you reply, and if you want to have an off-topic
conversation, please take it off-list.
And finally, as one of the list administrators I am in a position of power
when it comes to the rules of this list and the CoC. While I'm one of the
judges on when someone has violated the CoC, I purposefully try not to play
the role of police to avoid bias and abuse of power. What that means is
that I never personally lodge a CoC complaint against anyone. That means
that if you feel someone is being abusive here you cannot rely on list
admins noticing and doing something about it. If you feel someone has
continuously been abusive on this list and violating the CoC then you must
email the list admins about it if you wish to see action taken (all
communications are kept private among the admins). Now I'm not asking
people to email us on every small infraction (as I said above, try to give
everyone a break knowing we all have bad days), but if you notice a pattern
then you need to speak up if you would like to see something change.
When I started my month off I thought that maybe if I only read this
mailing list once a week that the frequency would be low enough that I
could handle the stress of being both a participant and admin who is
ultimately responsible for the behaviour here, but I'm afraid that isn't
going to cut it. What I don't think people realize is that I don't take my
responsibility as admin lightly; any time anyone acts rudely I take it
personally like I somehow failed by letting the atmosphere and discourse on
this list become what it is. Because of this I'm afraid I need to mute this
mailing list for the rest of my vacation from volunteering in the Python
community after I send this email. I personally hope people do take the
time to read this email and reflect upon how they conduct themselves on
this mailing list -- and maybe on other lists as well -- so that when I
attempt to come back in November I don't have to permanent stop being a
participant on this list and simply become an admin for this list to
prevent complete burn-out for me in the Python community (and I know this
last sentence sounds dramatic, but I'm being serious; the irony of
receiving the Frank Willison award the same year I'm having to contemplate
fundamentally shifting how I engage with the community to not burn out is
not lost on me).
-Brett
Sometimes I find myself in need of this nice operator that I used back in
the days when I was programming in .NET, essentially an expression
>>> expr ?? instead
should return expr when it `is not None` and `instead` otherwise.
A piece of code that I just wrote, you can see a use case:
def _sizeof(self, context):
if self.totalsizeof is not None:
return self.totalsizeof
else:
raise SizeofError("cannot calculate size")
With the oprator it would just be
def _sizeof(self, context):
return self.totalsizeof ?? raise SizeofError("cannot calculate
size")
pozdrawiam,
Arkadiusz Bulski
Hello all,
I would be happy to see a somewhat more general and user friendly
version of string.translate function.
It could work this way:
string.newtranslate(file_with_table, Drop=True, Dec=True)
So the parameters:
1. "file_with_table" : a text file with table in following format:
#[In] [Out]
97 {65}
98 {66}
99 {67}
100 {}
...
110 {110}
Notes:
All values are decimal or hex (to switch between parsing format use
Dec parameter)
As it turned out from my last discussion, majority prefers hex notation,
so I am not in mainstream with my decimal notation here, but both
should be supported.
Empty [Out] value {} means that the character will be deleted.
2. "Drop = True" this will set the default behavior for those values
which are NOT in the table.
For Drop = True: all values not defined in table set to [out] = {},
and be deleted.
For Drop=False: all values not defined in table set [out] = [in], so
those remain as is.
3. Dec= True : parsing format Decimal/hex. I use decimal everywhere.
Further thoughts: for 8-bit strings this should be simple to implement
I think. For 16-bit of course
there is issue of memory usage for lookup tables, but the gurus could
probably optimise it.
E.g. at the parsing stage it is not necessary to build the lookup
table for whole 16-bit range of course,
but take only values till the largest ordinal present in the table file.
About the format of table file: I suppose many users would want also
to define characters directly, I am not sure
if it is really needed, but if so, additional brackets or escape char
could be used, like this for example:
a {A}
\98 {\66}
\99 {\67}
but as said I don't like very much the idea and would be OK for me to
use numeric values only.
So approximately I see it.
Feel free to share thoughts or criticise.
Mikhail
Hi folks,
After the recent discussions of PEP 505's null-coalescing operator
(and the significant confusion around why anyone would ever want a
feature like that), I was inspired to put together a competing
proposal that focuses more on defining a new "existence checking"
protocol that generalises the current practicises of:
* obj is not None (many different use cases)
* obj is not Ellipsis (in multi-dimensional slicing)
* obj is not NotImplemented (in operand coercion)
* math.isnan(value)
* cmath.isnan(value)
* decimal.getcontext().is_nan(value)
Given that protocol as a basis, it then proceeds to define "?then" and
"?else" as existence checking counterparts to the truth-checking "and"
and "or", as well as "?.", "?[]" and "?=" as abbreviations for
particular idiomatic uses of "?then" and "?else".
I think this approach frames the discussion in a more productive way,
as it gives us a series of questions to consider in order where a
collective answer of "No" at any point would be enough to kill this
particular proposal (or parts of it), but precisely *where* we say
"No" will determine which future alternatives might be worth
considering:
1. Do we collectively agree that "existence checking" is a useful
general concept that exists in software development and is distinct
from the concept of "truth checking"?
2. Do we collectively agree that the Python ecosystem would benefit
from an existence checking protocol that permits generalisation of
algorithms (especially short circuiting ones) across different "data
missing" indicators, including those defined in the language
definition, the standard library, and custom user code?
3. Do we collectively agree that it would be easier to use such a
protocol effectively if existence-checking equivalents to the
truth-checking "and" and "or" control flow operators were available?
Only if we have at least some level of consensus on the above
questions regarding whether or not this is a conceptual modeling
problem worth addressing at the language level does it then make sense
to move on to the more detailed questions regarding the specific
proposed *solution* to the problem in the PEP:
4. Do we collectively agree that "?then" and "?else" would be
reasonable spellings for such operators?
5a. Do we collectively agree that "access this attribute only if the
object exists" would be a particularly common use case for such
operators?
5b. Do we collectively agree that "access this subscript only if the
object exists" would be a particularly common use case for such
operators?
5c. Do we collectively agree that "bind this value to this target only
if the value currently bound to the target nominally doesn't exist"
would be a particularly common use case for such operators?
6a. Do we collectively agree that 'obj?.attr' would be a reasonable
spelling for "access this attribute only if the object exists"?
6b. Do we collectively agree that 'obj?[expr]' would be a reasonable
spelling for "access this subscript only if the object exists"?
6c. Do we collectively agree that 'target ?= expr' would be a
reasonable spelling for "bind this value to this target only if the
value currently bound to the target nominally doesn't exist"?
To be clear, this would be a *really* big addition to the language
that would have significant long term ramifications for how the
language gets taught to new developers.
At the same time, asking whether or not an object represents an
absence of data rather than the truth of a proposition seems to me
like a sufficiently common problem in a wide enough variety of domains
that it may be worth elevating to the level of giving it dedicated
syntactic support.
Regards,
Nick.
Rendered HTML version: https://www.python.org/dev/peps/pep-0531/
===============================
PEP: 531
Title: Existence checking operators
Version: $Revision$
Last-Modified: $Date$
Author: Nick Coghlan <ncoghlan(a)gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 25-Oct-2016
Python-Version: 3.7
Post-History: 28-Oct-2016
Abstract
========
Inspired by PEP 505 and the related discussions, this PEP proposes the addition
of two new control flow operators to Python:
* Existence-checking precondition ("exists-then"): ``expr1 ?then expr2``
* Existence-checking fallback ("exists-else"): ``expr1 ?else expr2``
as well as the following abbreviations for common existence checking
expressions and statements:
* Existence-checking attribute access:
``obj?.attr`` (for ``obj ?then obj.attr``)
* Existence-checking subscripting:
``obj?[expr]`` (for ``obj ?then obj[expr]``)
* Existence-checking assignment:
``value ?= expr`` (for ``value = value ?else expr``)
The common ``?`` symbol in these new operator definitions indicates that they
use a new "existence checking" protocol rather than the established
truth-checking protocol used by if statements, while loops, comprehensions,
generator expressions, conditional expressions, logical conjunction, and
logical disjunction.
This new protocol would be made available as ``operator.exists``, with the
following characteristics:
* types can define a new ``__exists__`` magic method (Python) or
``tp_exists`` slot (C) to override the default behaviour. This optional
method has the same signature and possible return values as ``__bool__``.
* ``operator.exists(None)`` returns ``False``
* ``operator.exists(NotImplemented)`` returns ``False``
* ``operator.exists(Ellipsis)`` returns ``False``
* ``float``, ``complex`` and ``decimal.Decimal`` will override the existence
check such that ``NaN`` values return ``False`` and other values (including
zero values) return ``True``
* for any other type, ``operator.exists(obj)`` returns True by default. Most
importantly, values that evaluate to False in a truth checking context
(zeroes, empty containers) will still evaluate to True in an existence
checking context
Relationship with other PEPs
============================
While this PEP was inspired by and builds on Mark Haase's excellent work in
putting together PEP 505, it ultimately competes with that PEP due to
significant differences in the specifics of the proposed syntax and semantics
for the feature.
It also presents a different perspective on the rationale for the change by
focusing on the benefits to existing Python users as the typical demands of
application and service development activities are genuinely changing. It
isn't an accident that similar features are now appearing in multiple
programming languages, and while it's a good idea for us to learn from how other
language designers are handling the problem, precedents being set elsewhere
are more relevant to *how* we would go about tackling this problem than they
are to whether or not we think it's a problem we should address in the first
place.
Rationale
=========
Existence checking expressions
------------------------------
An increasingly common requirement in modern software development is the need
to work with "semi-structured data": data where the structure of the data is
known in advance, but pieces of it may be missing at runtime, and the software
manipulating that data is expected to degrade gracefully (e.g. by omitting
results that depend on the missing data) rather than failing outright.
Some particularly common cases where this issue arises are:
* handling optional application configuration settings and function parameters
* handling external service failures in distributed systems
* handling data sets that include some partial records
It is the latter two cases that are the primary motivation for this PEP - while
needing to deal with optional configuration settings and parameters is a design
requirement at least as old as Python itself, the rise of public cloud
infrastructure, the development of software systems as collaborative networks
of distributed services, and the availability of large public and private data
sets for analysis means that the ability to degrade operations gracefully in
the face of partial service failures or partial data availability is becoming
an essential feature of modern programming environments.
At the moment, writing such software in Python can be genuinely awkward, as
your code ends up littered with expressions like:
* ``value1 = expr1.field.of.interest if expr1 is not None else None``
* ``value2 = expr2["field"]["of"]["interest"] if expr2 is not None else None``
* ``value3 = expr3 if expr3 is not None else expr4 if expr4 is not
None else expr5``
If these are only occasional, then expanding out to full statement forms may
help improve readability, but if you have 4 or 5 of them in a row (which is a
fairly common situation in data transformation pipelines), then replacing them
with 16 or 20 lines of conditional logic really doesn't help matters.
Expanding the three examples above that way hopefully helps illustrate that::
_expr1 = expr1
if _expr1 is not None:
value1 = _expr1.field.of.interest
else:
value1 = None
_expr2 = expr2
if _expr2 is not None:
value2 = _expr2["field"]["of"]["interest"]
else:
value2 = None
_expr3 = expr3
if _expr3 is not None:
value3 = _expr3
else:
_expr4 = expr4
if _expr4 is not None:
value3 = _expr4
else:
value3 = expr5
The combined impact of the proposals in this PEP is to allow the above sample
expressions to instead be written as:
* ``value1 = expr1?.field.of.interest``
* ``value2 = expr2?["field"]["of"]["interest"]``
* ``value3 = expr3 ?else expr4 ?else expr5``
In these forms, almost all of the information presented to the reader is
immediately relevant to the question "What does this code do?", while the
boilerplate code to handle missing data by passing it through to the output
or falling back to an alternative input, has shrunk to two uses of the ``?``
symbol and two uses of the ``?else`` keyword.
In the first two examples, the 31 character boilerplate clause
`` if exprN is not None else None`` (minimally 27 characters for a single letter
variable name) has been replaced by a single ``?`` character, substantially
improving the signal-to-pattern-noise ratio of the lines (especially if it
encourages the use of more meaningful variable and field names rather than
making them shorter purely for the sake of expression brevity).
In the last example, two instances of the 21 character boilerplate,
`` if exprN is not None`` (minimally 17 characters) are replaced with single
characters, again substantially improving the signal-to-pattern-noise ratio.
Furthermore, each of our 5 "subexpressions of potential interest" is included
exactly once, rather than 4 of them needing to be duplicated or pulled out
to a named variable in order to first check if they exist.
The existence checking precondition operator is mainly defined to provide a
clear conceptual basis for the existence checking attribute access and
subscripting operators:
* ``obj?.attr`` is roughly equivalent to ``obj ?then obj.attr``
* ``obj?[expr]``is roughly equivalent to ``obj ?then obj[expr]``
The main semantic difference between the shorthand forms and their expanded
equivalents is that the common subexpression to the left of the existence
checking operator is evaluated only once in the shorthand form (similar to
the benefit offered by augmented assignment statements).
Existence checking assignment
-----------------------------
Existence-checking assignment is proposed as a relatively straightforward
expansion of the concepts in this PEP to also cover the common configuration
handling idiom:
* ``value = value if value is not None else expensive_default()``
by allowing that to instead be abbreviated as:
* ``value ?= expensive_default()``
This is mainly beneficial when the target is a subscript operation or
subattribute, as even without this specific change, the PEP would still
permit this idiom to be updated to:
* ``value = value ?else expensive_default()``
The main argument *against* adding this form is that it's arguably ambiguous
and could mean either:
* ``value = value ?else expensive_default()``; or
* ``value = value ?then value.subfield.of.interest``
The second form isn't at all useful, but if this concern was deemed significant
enough to address while still keeping the augmented assignment feature,
the full keyword could be included in the syntax:
* ``value ?else= expensive_default()``
Alternatively, augmented assignment could just be dropped from the current
proposal entirely and potentially reconsidered at a later date.
Existence checking protocol
---------------------------
The existence checking protocol is including in this proposal primarily to
allow for proxy objects (e.g. local representations of remote resources) and
mock objects used in testing to correctly indicate non-existence of target
resources, even though the proxy or mock object itself is not None.
However, with that protocol defined, it then seems natural to expand it to
provide a type independent way of checking for ``NaN`` values in numeric types
- at the moment you need to be aware of the exact data type you're working with
(e.g. builtin floats, builtin complex numbers, the decimal module) and use the
appropriate operation (e.g. ``math.isnan``, ``cmath.isnan``,
``decimal.getcontext().is_nan()``, respectively)
Similarly, it seems reasonable to declare that the other placeholder builtin
singletons, ``Ellipsis`` and ``NotImplemented``, also qualify as objects that
represent the absence of data moreso than they represent data.
Proposed symbolic notation
--------------------------
Python has historically only had one kind of implied boolean context: truth
checking, which can be invoked directly via the ``bool()`` builtin. As this PEP
proposes a new kind of control flow operation based on existence checking rather
than truth checking, it is considered valuable to have a reminder directly
in the code when existence checking is being used rather than truth checking.
The mathematical symbol for existence assertions is U+2203 'THERE EXISTS': ``∃``
Accordingly, one possible approach to the syntactic additions proposed in this
PEP would be to use that already defined mathematical notation:
* ``expr1 ∃then expr2``
* ``expr1 ∃else expr2``
* ``obj∃.attr``
* ``obj∃[expr]``
* ``target ∃= expr``
However, there are two major problems with that approach, one practical, and
one pedagogical.
The practical problem is the usual one that most keyboards don't offer any easy
way of entering mathematical symbols other than those used in basic arithmetic
(even the symbols appearing in this PEP were ultimately copied & pasted
from [3]_ rather than being entered directly).
The pedagogical problem is that the symbols for existence assertions (``∃``)
and universal assertions (``∀``) aren't going to be familiar to most people
the way basic arithmetic operators are, so we wouldn't actually be making the
proposed syntax easier to understand by adopting ``∃``.
By contrast, ``?`` is one of the few remaining unused ASCII punctuation
characters in Python's syntax, making it available as a candidate syntactic
marker for "this control flow operation is based on an existence check, not a
truth check".
Taking that path would also have the advantage of aligning Python's syntax
with corresponding syntax in other languages that offer similar features.
Drawing from the existing summary in PEP 505 and the Wikipedia articles on
the "safe navigation operator [1]_ and the "null coalescing operator" [2]_,
we see:
* The ``?.`` existence checking attribute access syntax precisely aligns with:
* the "safe navigation" attribute access operator in C# (``?.``)
* the "optional chaining" operator in Swift (``?.``)
* the "safe navigation" attribute access operator in Groovy (``?.``)
* the "conditional member access" operator in Dart (``?.``)
* The ``?[]`` existence checking attribute access syntax precisely aligns with:
* the "safe navigation" subscript operator in C# (``?[]``)
* the "optional subscript" operator in Swift (``?[].``)
* The ``?else`` existence checking fallback syntax semantically aligns with:
* the "null-coalescing" operator in C# (``??``)
* the "null-coalescing" operator in PHP (``??``)
* the "nil-coalescing" operator in Swift (``??``)
To be clear, these aren't the only spelling of these operators used in other
languages, but they're the most common ones, and the ``?`` symbol is the most
common syntactic marker by far (presumably prompted by the use of ``?`` to
introduce the "then" clause in C-style conditional expressions, which many
of these languages also offer).
Proposed keywords
-----------------
Given the symbolic marker ``?``, it would be syntactically unambiguous to spell
the existence checking precondition and fallback operations using the same
keywords as their truth checking counterparts:
* ``expr1 ?and expr2`` (instead of ``expr1 ?then expr2``)
* ``expr1 ?or expr2`` (instead of ``expr1 ?else expr2``)
However, while syntactically unambiguous when written, this approach makes
the code incredibly hard to *pronounce* (What's the pronunciation of "?"?) and
also hard to *describe* (given reused keywords, there's no obvious shorthand
terms for "existence checking precondition (?and)" and "existence checking
fallback (?or)" that would distinguish them from "logical conjunction (and)"
and "logical disjunction (or)").
We could try to encourage folks to pronounce the ``?`` symbol as "exists",
making the shorthand names the "exists-and expression" and the
"exists-or expression", but there'd be no way of guessing those names purely
from seeing them written in a piece of code.
Instead, this PEP takes advantage of the proposed symbolic syntax to introduce
a new keyword (``?then``) and borrow an existing one (``?else``) in a way
that allows people to refer to "then expressions" and "else expressions"
without ambiguity.
These keywords also align well with the conditional expressions that are
semantically equivalent to the proposed expressions.
For ``?else`` expressions, ``expr1 ?else expr2`` is equivalent to::
_lhs_result = expr1
_lhs_result if operator.exists(_lhs_result) else expr2
Here the parallel is clear, since the ``else expr2`` appears at the end of
both the abbreviated and expanded forms.
For ``?then`` expressions, ``expr1 ?then expr2`` is equivalent to::
_lhs_result = expr1
expr2 if operator.exists(_lhs_result) else _lhs_result
Here the parallel isn't as immediately obvious due to Python's traditionally
anonymous "then" clauses (introduced by ``:`` in ``if`` statements and suffixed
by ``if`` in conditional expressions), but it's still reasonably clear as long
as you're already familiar with the "if-then-else" explanation of conditional
control flow.
Risks and concerns
==================
Readability
-----------
Learning to read and write the new syntax effectively mainly requires
internalising two concepts:
* expressions containing ``?`` include an existence check and may short circuit
* if ``None`` or another "non-existent" value is an expected input, and the
correct handling is to propagate that to the result, then the existence
checking operators are likely what you want
Currently, these concepts aren't explicitly represented at the language level,
so it's a matter of learning to recognise and use the various idiomatic
patterns based on conditional expressions and statements.
Magic syntax
------------
There's nothing about ``?`` as a syntactic element that inherently suggests
``is not None`` or ``operator.exists``. The main current use of ``?`` as a
symbol in Python code is as a trailing suffix in IPython environments to
request help information for the result of the preceding expression.
However, the notion of existence checking really does benefit from a pervasive
visual marker that distinguishes it from truth checking, and that calls for
a single-character symbolic syntax if we're going to do it at all.
Conceptual complexity
---------------------
This proposal takes the currently ad hoc and informal concept of "existence
checking" and elevates it to the status of being a syntactic language feature
with a clearly defined operator protocol.
In many ways, this should actually *reduce* the overall conceptual complexity
of the language, as many more expectations will map correctly between truth
checking with ``bool(expr)`` and existence checking with
``operator.exists(expr)`` than currently map between truth checking and
existence checking with ``expr is not None`` (or ``expr is not NotImplemented``
in the context of operand coercion, or the various NaN-checking operations
in mathematical libraries).
As a simple example of the new parallels introduced by this PEP, compare::
all_are_true = all(map(bool, iterable))
at_least_one_is_true = any(map(bool, iterable))
all_exist = all(map(operator.exists, iterable))
at_least_one_exists = any(map(operator.exists, iterable))
Design Discussion
=================
Subtleties in chaining existence checking expressions
-----------------------------------------------------
Similar subtleties arise in chaining existence checking expressions as already
exist in chaining logical operators: the behaviour can be surprising if the
right hand side of one of the expressions in the chain itself returns a
value that doesn't exist.
As a result, ``value = arg1 ?then f(arg1) ?else default()`` would be dubious for
essentially the same reason that ``value = cond and expr1 or expr2`` is dubious:
the former will evaluate ``default()`` if ``f(arg1)`` returns ``None``, just
as the latter will evaluate ``expr2`` if ``expr1`` evaluates to ``False`` in
a boolean context.
Ambiguous interaction with conditional expressions
--------------------------------------------------
In the proposal as currently written, the following is a syntax error:
* ``value = f(arg) if arg ?else default``
While the following is a valid operation that checks a second condition if the
first doesn't exist rather than merely being false:
* ``value = expr1 if cond1 ?else cond2 else expr2``
The expression chaining problem described above means that the argument can be
made that the first operation should instead be equivalent to:
* ``value = f(arg) if operator.exists(arg) else default``
requiring the second to be written in the arguably clearer form:
* ``value = expr1 if (cond1 ?else cond2) else expr2``
Alternatively, the first form could remain a syntax error, and the existence
checking symbol could instead be attached to the ``if`` keyword:
* ``value = expr1 if? cond else expr2``
Existence checking in other truth-checking contexts
---------------------------------------------------
The truth-checking protocol is currently used in the following syntactic
constructs:
* logical conjunction (and-expressions)
* logical disjunction (or-expressions)
* conditional expressions (if-else expressions)
* if statements
* while loops
* filter clauses in comprehensions and generator expressions
In the current PEP, switching from truth-checking with ``and`` and ``or`` to
existence-checking is a matter of substituting in the new keywords, ``?then``
and ``?else`` in the appropriate places.
For other truth-checking contexts, it proposes either importing and
using the ``operator.exists`` API, or else continuing with the current idiom
of checking specifically for ``expr is not None`` (or the context appropriate
equivalent).
The simplest possible enhancement in that regard would be to elevate the
proposed ``exists()`` API from an operator module function to a new builtin
function.
Alternatively, the ``?`` existence checking symbol could be supported as a
modifier on the ``if`` and ``while`` keywords to indicate the use of an
existence check rather than a truth check.
However, it isn't at all clear that the potential consistency benefits gained
for either suggestion would justify the additional disruption, so they've
currently been omitted from the proposal.
Defining expected invariant relations between ``__bool__`` and ``__exists__``
-----------------------------------------------------------------------------
The PEP currently leaves the definition of ``__bool__`` on all existing types
unmodified, which ensures the entire proposal remains backwards compatible,
but results in the following cases where ``bool(obj)`` returns ``True``, but
the proposed ``operator.exists(obj)`` would return ``False``:
* ``NaN`` values for ``float``, ``complex``, and ``decimal.Decimal``
* ``Ellipsis``
* ``NotImplemented``
The main argument for potentially changing these is that it becomes easier to
reason about potential code behaviour if we have a recommended invariant in
place saying that values which indicate they don't exist in an existence
checking context should also report themselves as being ``False`` in a truth
checking context.
Failing to define such an invariant would lead to arguably odd outcomes like
``float("NaN") ?else 0.0`` returning ``0.0`` while ``float("NaN") or 0.0``
returns ``NaN``.
Limitations
===========
Arbitrary sentinel objects
--------------------------
This proposal doesn't attempt to provide syntactic support for the "sentinel
object" idiom, where ``None`` is a permitted explicit value, so a
separate sentinel object is defined to indicate missing values::
_SENTINEL = object()
def f(obj=_SENTINEL):
return obj if obj is not _SENTINEL else default_value()
This could potentially be supported at the expense of making the existence
protocol definition significantly more complex, both to define and to use:
* at the Python layer, ``operator.exists`` and ``__exists__`` implementations
would return the empty tuple to indicate non-existence, and otherwise return
a singleton tuple containing a reference to the object to be used as the
result of the existence check
* at the C layer, ``tp_exists`` implementations would return NULL to indicate
non-existence, and otherwise return a `PyObject *` pointer as the
result of the existence check
Given that change, the sentinel object idiom could be rewritten as::
class Maybe:
SENTINEL = object()
def __init__(self, value):
self._result = (value,) is value is not self.SENTINEL else ()
def __exists__(self):
return self._result
def f(obj=Maybe.SENTINEL):
return Maybe(obj) ?else default_value()
However, I don't think cases where the 3 proposed standard sentinel values (i.e.
``None``, ``Ellipsis`` and ``NotImplemented``) can't be used are going to be
anywhere near common enough for the additional protocol complexity and the loss
of symmetry between ``__bool__`` and ``__exists__`` to be worth it.
Specification
=============
The Abstract already gives the gist of the proposal and the Rationale gives
some specific examples. If there's enough interest in the basic idea, then a
full specification will need to provide a precise correspondence between the
proposed syntactic sugar and the underlying conditional expressions that is
sufficient to guide the creation of a reference implementation.
...TBD...
Implementation
==============
As with PEP 505, actual implementation has been deferred pending in-principle
interest in the idea of adding these operators - the implementation isn't
the hard part of these proposals, the hard part is deciding whether or not
this is a change where the long term benefits for new and existing Python users
outweigh the short term costs involved in the wider ecosystem (including
developers of other implementations, language curriculum developers, and
authors of other Python related educational material) adjusting to the change.
...TBD...
References
==========
.. [1] Wikipedia: Safe navigation operator
(https://en.wikipedia.org/wiki/Safe_navigation_operator)
.. [2] Wikipedia: Null coalescing operator
(https://en.wikipedia.org/wiki/Null_coalescing_operator)
.. [3] FileFormat.info: Unicode Character 'THERE EXISTS' (U+2203)
(http://www.fileformat.info/info/unicode/char/2203/index.htm)
Copyright
=========
This document has been placed in the public domain under the terms of the
CC0 1.0 license: https://creativecommons.org/publicdomain/zero/1.0/
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
--
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
Ken has made what I consider a very reasonable suggestion, to introduce
SI prefixes to Python syntax for numbers. For example, typing 1K will be
equivalent to 1000.
However, there are some complexities that have been glossed over.
(1) Are the results floats, ints, or something else?
I would expect that 1K would be int 1000, not float 1000. But what about
fractional prefixes, like 1m? Should that be a float or a decimal?
If I write 7981m I would expect 7.981, not 7.9809999999999999, so maybe
I want a decimal float, not a binary float?
Actually, what I would really want is for the scale factor to be tracked
separately. If I write 7981m * 1M, I should end up with 7981000 as an
int, not a float. Am I being unreasonable?
Obviously if I write 1.1K then I'm expecting a float. So I'm not
*entirely* unreasonable :-)
(2) Decimal or binary scale factors?
The SI units are all decimal, and I think if we support these, we should
insist that K == 1000, not 1024. For binary scale factors, there is the
IEC standard:
http://physics.nist.gov/cuu/Units/binary.html
which defines Ki = 2**10, Mi = 2**20, etc. (Fortunately this doesn't
have to deal with fractional prefixes.) So it would be easy enough to
support them as well.
(3) µ or u, k or K?
I'm going to go to the barricades to fight for the real SI prefixes µ
and k to be supported. If people want to support the common fakes u and
K as well, that's fine, I have no objection, but I think that its
important to support the actual prefixes too.
(Python 3 assumes UTF-8 as the default encoding, so it shouldn't cause
any technical difficulties to support µ as syntax. The political
difficulties though...)
(4) What about E?
E is tricky if we want 1E to be read as the integer 10**18, because it
matches the floating point syntax 1E (which is currently a syntax
error). So there's a nasty bit of ambiguity where it may be unclear
whether or not 1E is intended as an int or an incomplete float, and then
there's 1E1E which might be read as 1E1*10**18 or as just an error.
Replacing E with (say) X is risky. The two largest current SI prefixes
are Z and Y, it seems very likely that the next one added (if that ever
happens) will be X. Actually, using any other letter risks clashing with
a future expansion of the SI prefixes.
(5) What about other numeric types?
Just because there's no syntactic support for Fraction and Decimal
shouldn't mean we can't use these scale factors with them.
(6) What happens to int(), float() etc?
I wouldn't want int("23K") to suddenly change from being an error to
returning 23000. Presumably we would want int to take an optional
argument to allow the interpretation of scale factors.
This gives us an advantage: int("23E", scale=True) is unambiguously an
int, and we can ignore the fact that it looks like a float.
(7) What about repr() and str()?
I don't think that the repr() or str() of numeric types should change.
But perhaps format() could grow some new codes to display numbers using
either the most obvious scale factor, or some specific scale factor.
* * *
This leads to my first proposal: require an explicit numeric prefix on
numbers before scale factors are allowed, similar to how we treat
non-decimal bases.
8M # remains a syntax error
0s8M # unambiguously an int with a scale factor of M = 10**6
0s1E1E # a float 1E1 with a scale factor of E = 10**18
0s1.E # a float 1. with a scale factor of E, not an exponent
int('8M') # remains a ValueError
int('0s8M', base=0) # returns 8*10**6
Or if that's too heavy (two whole characters, plus the suffix!) perhaps
we could have a rule that the suffix must follow the final underscore
of the number:
8_M # int 8*10*6
123_456_789_M # int 123456789*10**6
123_M_456 # still an error
8._M # float 8.0*10**6
int() and float() take a keyword only argument to allow a scale factor
when converting from strings:
int("8_M") # remains an error
int("8_M", scale=True) # allowed
This solves the problem with E and floats. Its only a scale factor if it
immediately follows the final underscore in the float, otherwise it is
the regular exponent sign.
Proposal number two: don't make any changes to the syntax, but treat
these as *literally* numeric scale factors. Add a simple module to the
std lib defining the various factors:
k = kilo = 10**3
M = mega = 10**6
G = giga = 10**9
etc. and then allow the user to literally treat them as scale factors by
multiplying:
from scaling import *
int_value = 8*M
float_value = 8.0*M
fraction_value = Fraction(1, 8)*M
decimal_value = Decimal("1.2345")*M
and so forth. The biggest advantage of this is that there is no
syntactic changes needed, it is completely backwards compatible, it
works with any numeric type and even non-numbers:
py> x = [None]*M
py> len(x)
1000000
You can even scale by multiple factors:
x = 8*M*K
Disadvantages: none I can think of.
(Some cleverness may be needed to have fractional scale values work with
both floats and Decimals, but that shouldn't be hard.)
--
Steve
We have a syntax to create strings with variables automatically inferred
from its context:
>>> name = "Guido"
>>> print(f'Hello {name}')
Hello Guido
Similarly, I'd like to suggest a similar feature for building dictionaries:
>>> foo = 1
>>> bar = 2
>>> {:bar, :foo}
{'bar': 1, 'foo', 2}
And a similar way to get the content from the dictionary into variables:
>>> values = {'bar': 1, 'foo', 2}
>>> {:bar, :foo} = values
>>> bar
1
>>> foo
2
The syntaxes used here are of course just to illustrate the concept and
I'm suggesting we must use those.
I personally find it kind of annoying when you have code like this:
x = A(1, B(2, 3))
and Python's error message looks like this:
TypeError: __init__() takes 1 positional argument but 2 were given
It doesn't give much of a clue to which `__init__` is being called. At all.
The idea: when showing the function name in an error like this, show the
fully qualified name, like:
TypeError: A.__init__() takes 1 positional argument but 2 were given
This would be MUCH more helpful!
Another related change would be to do the same thing in tracebacks:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in __init__
AssertionError
to:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in MyClass.__init__
AssertionError
which could make it easier to find where exactly an error originated.
--
Ryan (ライアン)
[ERROR]: Your autotools build scripts are 200 lines longer than your
program. Something’s wrong.
http://kirbyfan64.github.io/
The null-coalescing discussion made me think about the current ternary "x =
a if b else c" expression. In normal "if / else" clauses, the "else" is
optional. I propose doing the same thing with ternary expressions
(although I don't know what the result would be called, a "binary
expression"?)
The idea would be to allow this syntax:
x = a if b
Which would be equivalent to:
x = a if b else x
I think this would be useful syntax. In particular, I see it being useful
for default value checking, but can also be used to override the result of
particular corner cases from functions or methods..
I've seen a lot of syntax proposals recently that are based around
providing better ways of writing "one liner" styles of code.
Typically, the proposals seem to get into trouble because:
1. They duplicate things that can already be done, just not in a
single expression/statement.
2. They are seen as over-terse, which is not generally seen as a good
thing in Python.
However, looking at them from the point of view of someone working at
the interactive prompt, they can seem much more attractive. The
natural unit of interaction at the command line is the single line. To
the extent that (for example) fixing a mistake in a multi-line
construct at the command line is a real pain.
But these limitations are not inherent to Python - they are problems
with the interactive prompt, which is fairly basic[1]. So maybe it's
worth looking at the root issue, how to make the interactive prompt
easier to use[2]?
But that's something of a solved problem. IPython offers a rich
interactive environment, for people who find the limitations of the
standard interactive prompt frustrating. Would it be worth the
standard Python documentation promoting IPython for that role? Maybe
even, if IPython is available, allowing the user to configure Python
to use it by default as the interactive prompt (a bit like readline,
but I dislike the way you can't switch off readline integration if
it's installed)? Ideally, if IPython was more readily available, fewer
users would be frustrated with Python's existing multi-line
constructs. And those that were, would have the option of looking into
custom IPython magic commands, before being forced to request language
changes.
Thoughts?
Paul
[1] On the other hand, the interactive prompt is a huge part of what
makes Python so great - these days, when I have to code in languages
that don't have an interactive prompt, it drives me nuts. And even
those that do, typically don't have one as good as Python's (in spite
of the fact that this whole mail is about needing to improve the
Python REPL).
[2] My apologies to anyone whose proposal was *not* based around
interactive use cases. I'm assuming motives here left, right and
centre, so if what I'm saying isn't what you were intending, that's
fine. Treat this as an unrelated proposal.