On Fri, Dec 06, 2019 at 09:11:44AM -0400, Juancarlo Añez wrote:
[...]
> > Sure, but in this case, it isn't a fragment of a larger function, and
> > that's not what it looks like. If it looked like what you wrote, I would
> > understand it. But it doesn't, so I didn't really understand what it was
> > supposed to do, until I read the equivalent version using first/next.
> >
>
> Exactly my point.
Indeed, and I agree with that. But I still don't see what advantage
there is to having a `first` builtin which does so little. It's a really
thin wrapper around `next` that:
calls iter() on its iterable argument
supplies a default
and then calls next() with two arguments
I guess my question is asking you to justify adding a builtin rather
than just educating people how to use next effectively.
This is how I would implement the function in Python:
def first(iterable, default=None):
return next(iter(iterable), default)
But there's a major difference in behaviour depending on your input, and
one which is surely going to lead to bugs from people who didn't realise
that iterator arguments and iterable arguments will behave differently:
# non-iterator iterable
py> obj = [1, 2, 3, 4]
py> [first(obj) for __ in range(5)]
[1, 1, 1, 1, 1]
# iterator
py> obj = iter([1, 2, 3, 4])
py> [first(obj) for __ in range(5)]
[1, 2, 3, 4, None]
We could document the difference in behaviour, but it will still bite
people and surprise them.
We could, I guess, eliminate the difference by adding the ability to
peek ahead to the next value of an arbitrary iterator without consuming
that value. This would have to be done by the interpreter, not in Python
code, and it would open new complexities.
Consider a generator which yields values which depend, in some complex
and unpredicatable way, on *when* it is called. Say, the amount of disk
space available, or the number of records in a database, or the current
time. If I peek into the generator, I would see the time at the moment I
peeked:
now = get_current_time_generator()
peek(now) # returns 11:25:30am
Since peek can't literally see into the future, it cannot be otherwise.
But what happens when I call next?
time.sleep(60)
next(now) # what will this return?
There are two alternatives:
1. `next(now)` will return 11:25:30am, the same value that peek gave;
2. `next(now)` will return 11:27:30am, the current time.
Option 1 keeps the invariant that peeking will show you the next value
without advancing the iterable, but it violates the invariant that
`next(now)` yields the current time.
Option 2 keeps the `next` invariant, but violates the `peek` invariant.
Whichever option we choose, peeking into arbitrary iterators will break
somebody's expectations.
Bringing it back to `first`:
* It seems to me that `first` adds very little that `next` doesn't
already give us.
* The simple and obvious implementation of `first` would have a
troublesome difference in behaviour between iterator arguments
and non-iterator arguments.
* To eliminate that difference would require the ability to peek
ahead into arbitrary iterators, including generators, which is
a much bigger change, and equally troublesome.
--
Steven
PEP 479 (https://www.python.org/dev/peps/pep-0479/) changed the rules
around generators: if it would have leaked StopIteration, it instead
raises RuntimeError. This converts hard-to-debug premature termination
into easily-spotted exceptions, but it applies only to actual
generator functions.
Implementing a map-like function as a generator means you're safe:
def poof(x):
if x == 5: raise StopIteration # Leak!
return x * 3
def mymap(func, it):
for value in it:
yield func(value)
for n in mymap(poof, range(10)):
print(n)
Boom! RuntimeError that highlights the source of the StopIteration.
Similarly, using a list comprehension is safe:
[poof(n) for n in range(10)] # raises StopIteration
But using the built-in map() will silently terminate:
for n in map(poof, range(10)):
print(n)
I propose to grant PEP 479 semantics - namely, that a StopIteration
during the calling of the mapped function be translated into a
RuntimeError. Likewise for filter(), guarding the predicate function,
and all similar functions in itertools: accumulate, filterfalse,
takewhile/dropwhile, starmap, and any that I didn't notice.
ChrisA
On Wed, Dec 4, 2019 at 3:02 PM Guido van Rossum <guido(a)python.org> wrote:
> Fair enough. I’ll let the OP defend his use case.
>
The OP thinks that the case for wanting just the string for a first regex
match, or a verifiable default if there is no match, is way too common,
that the advice on the web is not very good (it should be "write a
findfirst() using next() over finditer()", and that novices default to
using findall(..)[0], which is troublesome.
The proposed implementation of a findfirst() would handle many common
cases, and be friendly to newcomers (why do I need to deal with a Match
object?), specially if the semantics are those of *findall()*:
next(iter(findall(...)), default=default)
BTW, a common function in extensions to *itertools* is *first():*
def first(seq, default=None):
return next(iter(seq), default= default)
That function, *first()*, would also be a nice addition in *itertools*, and
*findfirst()* could be implemented using it. *first()* avoids most use
cases needing to check if a sequence or iterator is empty before using a
default value. MHO is that *first()* deals with so many common cases that
it should be a builtin.
Note that the case for *findfirst()* is weaker if *first()* is available.
Yet *findfirst()* solves the bigger problem.
--
Juancarlo *Añez*
I've been tripped up a couple of time by a few Exception names being lower case both in code and tracebacks. In particular `error` (all lower case)
I tend to read `error` as a function instead of a class.
According to my non-scientific grep-foo, I find that ~ 300 descendant from Exceptions start with an upper case:
~/dev/cpython[master ✗] $ rg '^[ ]*class [A-Z][a-zA-Z]+\(' | grep Excep | wc -l
333
18 of those are `Error` uppercase:
~/dev/cpython[master ✗] $ rg '^[ ]*class Error\(' | wc -l
18
but 4 are lower case:
~/dev/cpython[master ✗] $ rg '^[ ]*class [a-z][a-zA-Z]+\(' | grep Exception
Lib/sre_constants.py:class error(Exception):
Lib/imaplib.py: class error(Exception): pass # Logical errors - debug required
Doc/faq/design.rst: class label(Exception): pass # declare a label
Lib/dbm/__init__.py:class error(Exception):
... there are probably others if they don't directly inherit from Exception of course.
My question is in two part:
1) For consistency would it make sens to try to capitalize – where possible – Exceptions names ? For example `error` to `Error` (keeping error as an alias for backward compatibility for now).
2) If (1), would it be ok to actually give a better name to some of those exceptions ? e.g: Lib/sre_constants.py:class error to CompileError for example.
Thanks.
I just found this code:
def get_product_item(jsonld_items):
for item in jsonld_items:
if item['@type'] == 'Product':
return item
else:
return {}
My argument is that the intent is clearer in:
def get_product_item(jsonld_items):
return first((item for item in jsonld_items if item['@type'] ==
'Product'), {})
As a reminder, first()'s definition in Python is:
def first(seq, default=None):
return next(iter(seq), default=default)
It could be optimized (implemented in C) if it makes it into the stdlib.
--
Juancarlo *Añez*
What if there was a general mechanism to allow operators to be implemented by user code that does not belong to the class?
If the name [e.g.] __operatorhook_or__ is defined anywhere in a module, within that module all calls to that operator are replaced with a two-step process:
call __operatorhook_or__(dict1, dict2)
if it returns a value, use that value
if it returns NotImplemented use the ordinary operator lookup process, i.e. dict1.__or__ and dict2.__ror___
[if __operatorhook_or__ is not defined, treat the same as if it had returned NotImplemented]
Then a user could simply do:
def __operatorhook_or__(obj1, obj2):
if isinstance(obj1, dict) and isinstance(obj2, dict):
return {**obj1, **obj2}
return NotImplemented
def __operatorhook_ior__(obj1, obj2):
if isinstance(obj1, dict):
obj1.update(obj2)
return obj1
return NotImplemented
Hi folks,
I have a few decades of experience with software engineering using
assembler, C, FORTRAN, Pascal, Ada and a fair number of other languages.
I've been using Python as well for a number of years now.
Both Pascal and Ada have "end" statements, which make the code (at least
for me :-) ) a lot more readable and maintainable, especially if the
code within if/while/for statements gets longer and nested deeper. These
statements exist in these languages for a good reason.
I had a look in the archives to see if someone else already proposed
this but couldn't find anything, so here goes.
Please let me know whether you think this idea is worth to convert to a PEP.
thanks,
Jan
Idea:
To include optional "end if", "end while", "end for" and "end <function
name>", "end <class name>" in the language. The Python interpreter would
test that any "end" matches with the correct corresponding statements
and have the correct indentation, in other words throw an error if you
would write "end if" where an "end while" should be. This will be a
great help preventing errors with procedural logic, that I now have to
painstakingly debug at runtime, especially after moving bits of code around.
Python editors would ideally highlight "if" and "end if" just like they
now highlight "if". By default different colors could be picked for "if"
and "end if", "while" and "end while" etc.
All this is completely optional of course, so it won't break any
existing code.
Advantages:
- better readability and maintainability
- help prevent procedural logic mistakes
- editors could use this for improved highlighting of code
Disadvantages:
- none :-) it's optional, just like the typing that was introduced in
Python 3
Code example:
def func (i: int, j: int, k: int) -> None:
if (i == 3):
while (i < 15):
i += 1
if (k == 8):
lots of other code (think a page or more)
lots of other code
here is more code <- to which if or while does this belong?
and a bit more
further code
The code could be rewritten like this:
def func (i: int, j: int, k: int) -> None:
if (i == 3):
while (i < 15):
i += 1
if (k == 8):
lots of other code (think a page or more)
lots of other code
end if
end while (writing "end if" would throw an error at parse time)
here is more code
and a bit more
end if
end func
further code
Looking forward to your feedback!
regards,
Jan
NEW ZEALAND
I suggest we find a way to close a mail thread. Obviously we can't stop
people from sending mails, but, like when a mod sees the topic has been
juiced out, he closes the topic which practically means:
if mailman sees mails being sent to a topic closed by a mod, it does not
relay it to the list.
It is normally difficult to say when a topic has been juiced out (the
essentials have been said, with no new inputs). One might close the topic
prematuraly. So, a time-based automatic closing might be better. Example:
After 5 days, a topic is closed. A mod can extend the closing though.
Abdur-Rahmaan Janhangeer
http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ
Mauritius
Well said!
On Wed, Dec 4, 2019 at 16:33 Juancarlo Añez <juancarlo.anez(a)gmail.com>
wrote:
>
>
> On Wed, Dec 4, 2019 at 3:02 PM Guido van Rossum <guido(a)python.org> wrote:
>
>> Fair enough. I’ll let the OP defend his use case.
>>
>
> The OP thinks that the case for wanting just the string for a first regex
> match, or a verifiable default if there is no match, is way too common,
> that the advice on the web is not very good (it should be "write a
> findfirst() using next() over finditer()", and that novices default to
> using findall(..)[0], which is troublesome.
>
> The proposed implementation of a findfirst() would handle many common
> cases, and be friendly to newcomers (why do I need to deal with a Match
> object?), specially if the semantics are those of *findall()*:
>
> next(iter(findall(...)), default=default)
>
> BTW, a common function in extensions to *itertools* is *first():*
>
> def first(seq, default=None):
> return next(iter(seq), default= default)
>
> That function, *first()*, would also be a nice addition in *itertools*,
> and *findfirst()* could be implemented using it. *first()* avoids most
> use cases needing to check if a sequence or iterator is empty before using
> a default value. MHO is that *first()* deals with so many common cases
> that it should be a builtin.
>
> Note that the case for *findfirst()* is weaker if *first()* is available.
> Yet *findfirst()* solves the bigger problem.
>
> --
> Juancarlo *Añez*
> tel:+58(414)901-2021
> skype:juancarloanez
>
--
--Guido (mobile)
A month ago I promised the Steering Council I'd review PEP 584 and promptly
got distracted by other things. But here it is, at last.
Some procedural things first: you (the PEP authors) should apply to the
Steering Council for a formal review. They most likely will assign one of
the SC members as a BDFL-Delegate who will guide the PEP to a resolution
(either acceptance or rejection, after any number of revisions). Don't wait
until you think the PEP is perfect! It doesn't have to be for a
BDFL-Delegate to be assigned.
Note that the SC elections have just started, and in a few weeks we'll have
a new SC. I won't be in it (I withdrew from the ballot). I don't know how
this will affect the chances of this PEP. (Also note that after an
election, a SC can overturn decisions by the previous SC, so I don't mean
this as a reason to hurry -- just something to consider.)
Now to the content of PEP 584.
## Should the operators be + and +=, or | and |= ?
I haven't read _all_ the posts on this topic, but I've seen quite a few,
and I still strongly favor | and |= . My reason for this is mainly that the
operation really does do a set union **on the keys**.
Also, + is already overloaded enough (numbers, strings, sequences) and we
don't need yet another, different meaning. The | operator should be known
to most users from sets (which are introduced before dicts in the official
Python tutorial).
I don't particularly care about learnability or discoverability in this
case -- I don't think beginners should be taught these operators as a major
tool in their toolbox (let them use .update()), and for anyone wondering
how to do something with a dict without access to the official docs,
there's always help(dict). The natural way of discovering these would be
through tutorials, reading of the library docs, or by reading other
people's code, not by trying random key combinations in the REPL.
## Does it matter that the operation is lossy, and not commutative in the
values?
Centithreads have been wasted on this topic, and I really don't care. I
**certainly** don't think that introducing a new operator or using a
different operator (<<, really?) is needed because of the operator's
asymmetry. A tutorial can show this in two or three lines, and anybody who
remembers these operators exists but has forgotten how they handle
conflicting values (and doesn't remember they correspond to.update()) can
type one line in the REPL to figure it out:
>>> {"a": 1} | {"a": 2}
{"a": 2}
>>>
## What about the Counter class?
The Counter class is an over-engineered contraption that has way too many
APIs to remember, and is quite inconsistent in what it tries to be. (When
does it delete zeros? Why does it sometimes discard negative values?) We
should not use it for guidance nor should we feel constrained by its design.
## What about performance?
Performance is not the only objective when using Python. Switching to
inplace operators (here |=) is a generally useful and well-known technique
(it also applies to string and list concatenation, for example).
## Other objections
The PEP does a good job of addressing pretty much everything that's been
brought up against it (in fact, perhaps it goes a little too far in
humoring obvious strawmen).
## Open questions
There's only one left in the PEP: "Should these operators be part of the
ABC Mapping API?"
No. This would require every Mapping implementation to add these. While you
might think that this could be addressed by adding concrete implementations
in the Mapping ABC, that doesn't work for |, since there's no API for
creating new instances.
It could be done for |=, by adding this to MutableMapping:
def __ior__(self, other):
self.update(other)
but without the ability to define __or__ in Mapping this seems
inconsistent. Mapping implementations are of course free to define __or__.
All in all I would recommend to the SC to go forward with this proposal,
targeting Python 3.9, assuming the operators are changed to | and |=, and
the PEP is brought more in line with the PEP editing guidelines from PEP 1
and PEP 12. (In particular, there are some suggested section headings there
that ought to be followed more closely. I'd be glad to give the authors
more guidance in this matter if they request it.)
--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-ca...>
