Hello,
This is a great PEP. It turns out to be applicable in a variety of scenarios.
Case in point: Matthew Rahtz and I are working on PEP 646: Variadic Generics (https://www.python.org/dev/peps/pep-0646/; discussion on typing-sig). It is a type system feature that allows specifying an arbitrary tuple of type variables instead of a single type variable.
We plan to use your proposed syntax to represent unpacking a tuple type. This would be analogous to `*` for unpacking a tuple value:
+ `Tensor[int, *Ts, str]` and `Tensor[*Ts1, *Ts2]`
+ such variadic classes would be declared as `class Tensor(Generic[T, *Ts, T2]):`
Questions:
1. Does PEP 637 support unpacking multiple `*` arguments?
- e.g., Tensor[int, *Ts, str, *Ts2]
2. Does PEP 637 allow a positional argument after a `*`?
- e.g., Generic[T, *Ts, T2]
PEP 637 says "Sequence unpacking is allowed inside subscripts", so it looks like these should be allowed (just as in function calls). But I wanted to confirm it explicitly since this is our core use case and there was no example with multiple sequences being unpacked.
3. We also wanted to ask - how's your implementation going?
We'll be starting implementation in typing.py soon. Since there's some overlap we wanted to make sure we're not duplicating your work, and that there won't be any merge conflicts later. Do you have a fork we might be able to get early access to? We're also targeting the 3.10 release for our implementation.
I'd be happy to provide additional details if needed.
Best,
Pradeep Kumar Srinivasan
Matthew Rahtz
Currently, python allows variable documentation via PEP 526
<https://www.python.org/dev/peps/pep-0526/>. For most functions with short
parameter lists that can fit in a reasonable column limit, I prefer the
traditional declaration style with Google-style doc strings:
*def connect_to_next_port(self, minimum: int) => int: *
"""Connects to the next available port.
Args:
minimum: A port value greater or equal to 1024.
Returns:
The new minimum port.
Raises:
ConnectionError: If no available port is found.
"""
...code...
However, when a signature gets too long, I prefer to list the parameters
vertically:
*def request(*
* method: Method, url: Str,*
* params: Dict = None, data: Dict = None, json: Str =
None, headers: Dict = None, cookies: Dict = None,
files: Dict = None, ...) => Response: """*
*Constructs and sends a Request*
* Args: ... """ *
In which case, it would be nice to in-line some documentation instead of
repeating the whole parameter list in the doc string. Something like:
*def request(*
* method: Method*
* #method for the new Request: ``GET``,``POST``, etc.*
* , url: Str*
* #URL for the request ,*
* params: Dict = None*
* ...**) => Response:*
* """**Constructs and sends a Request*
*"""*
Hi, all.
I am rewriting PEP 597 to introduce a new EncodingWarning, which
subclass of DeprecationWarning and used to warn about future default
encoding change.
But I don't think we can change the default encoding of
`io.TextIOWrapper` and built-in `open()` anytime soon. It is
disruptive change. It may take 10 or more years.
To ease the pain caused by "default encoding is not UTF-8 (almost)
only on Windows" (*), I came up with another idea. This idea is not
mutually exclusive with PEP 597, but I want to include it in the PEP
because both ideas use EncodingWarning.
(*) Imagine that a new Python user writes a text file with notepad.exe
(default encoding is UTF-8 without BOM already) or VS Code, and try to
read it in Jupyter Notebook. They will see UnicodeDecodeError. They
might not know about what encoding yet.
## 1. Add `io.open_text()`, builtin `open_text()`, and
`pathlib.Path.open_text()`.
All functions are same to `io.open()` or `Path.open()`, except:
* Default encoding is "utf-8".
* "b" is not allowed in the mode option.
These functions have two benefits:
* `open_text(filename)` is shorter than `open(filename,
encoding="utf-8")`. Its easy to type especially with autocompletion.
* Type annotation for returned value is simple than `open`. It is
always TextIOWrapper.
## 2. Change the default encoding of `pathlib.Path.read_text()`.
For convenience and consistency with `Path.open_text()`, change the
default encoding of `Path.read_text()` to "utf-8" with regular
deprecation period.
* Python 3.10: `Path.read_text()` emits EncodingWarning when the
encoding option is omitted.
* Python 3.13: `Path.read_text()` change the default encoding to "utf-8".
If PEP 597 is accepted, users can pass `encoding="locale"` instead of
`encoding=locale.getpreferredencoding(False)` when they need to use
locale encoding.
We might change more places where the default encoding is used. But it
should be done slowly and carefully.
---
How do you think about this idea? Is this worth enough to add a new
built-in function?
Regards,
--
Inada Naoki <songofacandy(a)gmail.com>
Sorry for posting multiple threads so quickly.
Microsoft provides UTF-8 code page for process. It can be enabled by
manifest file.
https://docs.microsoft.com/ja-jp/windows/uwp/design/globalizing/use-utf8-co…
How about providing Python binaris both of "UTF-8 version" and "ANSI version"?
This idea can provide a more smooth transition of the default encoding.
1. Provide UTF-8 version since Python 3.10
2. (Some years later) Recommend UTF-8 version
3. (Some years later) Provide only UTF-8 version
4. (Some years later, maybe) Change the default encoding
The upsides of this idea are:
* We don't need to emit a warning for `open(filename)`.
* We can see the download stats.
Especially, the last point is a huge advantage compared to current
UTF-8 mode (e.g. PYTHONUTF8=1).
We can know how many users need legacy behavior in new Python
versions. That is a very important information for us.
Of course, there are some downsides:
* Windows team needs to maintain more versions.
* More divisions for "Python on Windows" environment.
Regards,
--
Inada Naoki <songofacandy(a)gmail.com>
I've created a helper class in my own library that enhances the
existing dataclass:
a) __init__ accepts keyword-only arguments,
b) Optional[...] attribute without a specified default value would
default to None in __init__.
I think this could be useful in stdlib. I'm thinking a dataclass
decorator parameter like "init_kwonly" (default=False to provide
backward compatibility) that if True would implement this behavior.
Thoughts?
My previous thread is hijacked about "auto guessing" idea, so I split
this thread for pathlib.
Path.open() was added in Python 3.4. Path.read_text() and
Path.write_text() was added in Python 3.5.
Their history is shorter than built-in open(). Changing its default
encoding should be easier than built-in open and TextIOWrapper.
New default encodings are:
* read_text() default encoding is "utf-8-sig"
* write_text() default encoding is "utf-8"
* open() default encoding is "utf-8-sig" when mode is "r" or None,
"utf-8" otherwise.
Of course, we need a regular deprecation period.
When encoding is omitted, they emit DeprecationWarning (or
EncodingWarning which is a subclass of DeprecationWarning) in three
versions (Python 3.10~3.12).
How do you think this idea?
Should we "change all at once" rather than "step-by-step"?
Regards,
--
Inada Naoki <songofacandy(a)gmail.com>
Hi Ilya,
I'm not sure that this mailing list (Python-Dev) is the right place for
this discussion, I think that Python-Ideas (CCed) is the correct place.
For the benefit of Python-Ideas, I have left your entire post below, to
establish context.
[Ilya]
> I needed reversed(enumerate(x: list)) in my code, and have discovered
> that it wound't work. This is disappointing because operation is well
> defined.
It isn't really well-defined, since enumerate can operate on infinite
iterators, and you cannot reverse an infinite stream. Consider:
def values():
while True:
yield random.random()
a, b = reversed(enumerate(values())
What should the first pair of (a, b) be?
However, having said that, I think that your idea is not unreasonable.
`enumerate(it)` in the most general case isn't reversable, but if `it`
is reversable and sized, there's no reason why `enumerate(it)` shouldn't
be too.
My personal opinion is that this is a fairly obvious and straightforward
enhancement, one which (hopefully!) shouldn't require much, if any,
debate. I don't think we need a new class for this, I think enhancing
enumerate to be reversable if its underlying iterator is reversable
makes good sense.
But if you can show some concrete use-cases, especially one or two from
the standard library, that would help your case. Or some other languages
which offer this functionality as standard.
On the other hand, I think that there is a fairly lightweight work
around. Define a helper function:
def countdown(n):
while True:
yield n
n -= 1
then call it like this:
# reversed(enumerate(seq))
zip(countdown(len(seq)-1), reversed(seq)))
So it isn't terribly hard to work around this. But I agree that it would
be nice if enumerate encapsulated this for the caller.
One potentially serious question: what should `enumerate.__reversed__`
do when given a starting value?
reversed(enumerate('abc', 1))
Should that yield...?
# treat the start value as a start value
(1, 'c'), (0, 'b'), (-1, 'a')
# treat the start value as an end value
(3, 'c'), (2, 'b'), (1, 'a')
Something else?
My preference would be to treat the starting value as an ending value.
Steven
On Wed, Apr 01, 2020 at 08:45:34PM +0200, Ilya Kamenshchikov wrote:
> Hi,
>
> I needed reversed(enumerate(x: list)) in my code, and have discovered that
> it wound't work. This is disappointing because operation is well defined.
> It is also well defined for str type, range, and - in principle, but not
> yet in practice - on dictionary iterators - keys(), values(), items() as
> dictionaries are ordered now.
> It would also be well defined on any user type implementing __iter__,
> __len__, __reversed__ - think numpy arrays, some pandas dataframes, tensors.
>
> That's plenty of usecases, therefore I guess it would be quite useful to
> avoid hacky / inefficient solutions like described here:
> https://code.activestate.com/lists/python-list/706205/.
>
> If deemed useful, I would be interested in implementing this, maybe
> together with __reversed__ on dict keys, values, items.
>
> Best Regards,
> --
> Ilya Kamen
>
> -----------
> p.s.
>
> *Sketch* of what I am proposing:
>
> class reversible_enumerate:
>
> def __init__(self, iterable):
> self.iterable = iterable
> self.ctr = 0
>
> def __iter__(self):
> for e in self.iterable:
> yield self.ctr, e
> self.ctr += 1
>
> def __reversed__(self):
> try:
> ri = reversed(self.iterable)
> except Exception as e:
> raise Exception(
> "enumerate can only be reversed if iterable to
> enumerate can be reversed and has defined length."
> ) from e
>
> try:
> l = len(self.iterable)
> except Exception as e:
> raise Exception(
> "enumerate can only be reversed if iterable to
> enumerate can be reversed and has defined length."
> ) from e
>
> indexes = range(l-1, -1, -1)
> for i, e in zip(indexes, ri):
> yield i, e
>
> for i, c in reversed(reversible_enumerate("Hello World")):
> print(i, c)
>
> for i, c in reversed(reversible_enumerate([11, 22, 33])):
>
> print(i, c)
> _______________________________________________
> Python-Dev mailing list -- python-dev(a)python.org
> To unsubscribe send an email to python-dev-leave(a)python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NDDKDUD…
> Code of Conduct: http://python.org/psf/codeofconduct/
Hi, all.
I want to write type hints without worrying about runtime overhead.
Current best practice is:
```
from __future__ import annotations
import typing
if typing.TYPE_CHECKING:
import xxx # modules used only in type hints.
```
But it would be nice if I can avoid importing even "typing" module.
How about adding TYPE_CHECKING builtin that is False?
```
from __future__ import annotations
if TYPE_CHECKING:
from typing import Any, Optional
# We can use Any, Optional, etc here.
```
I wonder if we can make TYPE_CHECKING constant like True, False, and None.
But it will break existing `from typing import TYPE_CHECKING` codes.
Regards,
--
Inada Naoki <songofacandy(a)gmail.com>
Hello,
There're tons of projects which introduce alternative braces
(i.e. C-like) syntax for Python. Most of them are however not properly
documented, and definitely not spec'ed for what they do.
I wonder, does anyone here remember more or less formal proposal for
braces syntax? A "minimum viable product" criteria would be
support for lossless indent -> braces -> indent syntax roundtripping.
--
Best regards,
Paul mailto:pmiscml@gmail.com
Hi,
I've found myself typing too many time this:
pathlib.Path("some-dir-name").mkdir(parent=True, exist_ok=True)
Wouldn't be better to have a pathlib.Path.makedirs with parent/exist_ok set
to True by default?
Thanks
