As discussed earlier on the post on 'Circular Indexing', considering interpreting indices that lie outside the range 0 to n-1 modulo n for list indexing. Thus element n corresponds to element 0, n+1 to element 1 and so on. This has two consequences:
1.) Makes the Python convention of interpreting index -1 as the last element more logical since -1 is the same as n-1 when taken modulo n. If -1 is the first element then to be logically consistent we have to interpret index n as the element 0 as well!
Obviously this does allow certain errors to slip but if you are going to allow an index of -1 (indices that occur before 0) then you might just as well allow indices above n-1 to be logically consistent.
2.) If circular indexing is used then instead of using a double FOR loop to go through a loop twice we can iterate from 0 to 2n !
Although trivial for the case of one dimensional lists/arrays, cylindrical indexing (for 2D arrays) and toroidal indexing (2D, 3D and nD arrays) schemes might be worth exploring for n-dimensional NumPy arrays.
Circular and in general cylindrical or toroidal indexing schemes might simplify coding by reducing the number of FOR loops when iterating over nD arrays in NumPy.
Greetings list,
> Then I would suggest starting a new thread,
Long overdue i guess. My view on the topic is like
Mr Moore with the exception that i'm for native
capabilities to be in the stdlib. Since Zipapp does
not seem to go the C-extension route, i propose
a new tool that turn projects into independent executables
This solves the issue of Gui etc etc etc etc etc etc
Mr Moore's mail of Nov 24, 2020 has ton of ideas
and a lot of potential
Kind Regards,
Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://abdur-rahmaanj.github.io/>
github <https://github.com/Abdur-RahmaanJ>
Mauritius
I was reading the pyinstaller thread and had this idea but didn't want to
hijack.
Maybe a wild idea, and very possible totally impractical or hopelessly
complex, but: could the existing pypi infrastructure be leveraged into a
set of platform-specific app stores? Maybe maybe we could make it as simple
as a single line in a pyproject.toml file.
Imagine if all the end user does is install the store, and clicks the link
to the project app. The store takes care of the rest. The developer marks
their package as an installable program to be published to the store,
optionally specifying things like specific platforms if needed. All the
project website does for users to install is provide a url to their app in
the store.
Very very rough idea of how it might work: a store app API would provide an
installation GUI if desired, and the store would build the environment
needed for installing the package and dependencies. Once the specified
environment is built, pip would take care of installing the package as
usual.
If this is an awful idea feel free to just say so. I have given it no deep
thought and do not have the expertise to even think through what would be
needed to create such a thing.
Greetings list,
What do you think of adding PyInstaller as an official
part of CPython? Among the different native exports
options, PyInstaller holds a nice track of clean delivery.
Instead of the project battling for its survival as any other
project, if the community finds it useful enough, the devs
can get some peace of mind.
The stereotyped idea about languages like Python is that
they don't by default produce executables compared to let's
say Go. Adding the native ability to produce executables can
be a turning point for Python. In the process, PyInstaller might
become even better.
Sure it will need people who are well versed in at least 3 operating
systems to maintain the project. It will add to the testing load but
if it's worth it, it's worth it. Though for a good DevOps team, set up
is not a concern. However, having an executable makes the task easy
as you just ship a final product.
I don't exactly know how executables will interact with the WSGI protocol
but these concerns are for after and is an issue which does not concern
integrating a library per se.
Kind Regards,
Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://abdur-rahmaanj.github.io/>
github <https://github.com/Abdur-RahmaanJ>
Mauritius
I was thinking about the "Load JSON file as single line" thread from a bit
back, and had an idea for a neat solution, albeit one that has potential
stylistic issues.
Idea:
Create two new methods on pathlib.Path objects:
Path.load(loader, **kwargs)
and
Path.dump(dumper, obj, **kwargs)
Here, `loader` should be an object that implements a method `load(fileobj:
BinaryIO, *args, **kwargs) -> object`
and `dumper` should be an object that implements `dump(obj: object,
fileobj: BinaryIO, *args, **kwargs)`
This approach takes advantage of the standard `dump/load` pattern used by
json, pickle and other serialization libraries.
Example usage:
data = Path("/path/to/file.json").load(json)
Path("/dest.json").dump(json, my_obj, indent=2)
The implementation should look something like:
def load(self, loader, *args, **kwargs):
with self.open('rb') as fh:
return loader.load(fh, *args, **kwargs)
and the equivalent for `dump`
Reasons I like this:
------------------------
* Reading the code, the intent seems clear (generally unambiguous)
* It's explicit in naming the serialization method
* The code boilerplate is quite low (compared to context managers etc)
* Implementation is simple & decoupled
* Using other serialization is trivial:
my_path.load(json)
# vs.
my_path.load(pickle)
my_path.load(msgpack)
...
* Unlike other similar proposals, this doesn't force the solution to read
the entire file into memory before decoding (although optional support for
`loads`/`dumps` could be added if desired)
Reasons I don't like it:
-------------------------------
* Relying on the dump/load convention isn't something I've seen much of
elsewhere
* It may be hard to type-check (is this a concern?)
* We're just replacing 2 lines of code with one (previous discussions
around this topic seemed to suggest that consensus was that this pattern is
so common, that maybe this is a good-enough reason).
* The load/dump naming pattern clashes with pathlib's read/write naming.
I'm not concerned about exactly what these methods are called but this
feels like it's a side-effect of either: "read/write and dump/load being
slightly different operations", or "a tiny existing inconsistency in the
standard library" either way, the problem here is we're bringing these
existing things into the same module, which seems fairly minor.
Steve
I am excited about the potential of the new PEP 634-636 "match" statement to
match JSON data received by Python web applications. Often this JSON data is
in the form of structured dictionaries (TypedDicts) containing Lists and
other primitives (str, float, bool, None).
PEP 634-636 already contain the ability to match all of those underlying
data
types except for TypedDicts, so I'd like to explore what it might look
like to
match a TypedDict...
Consider an example web application that wants to provide a service to draw
shapes, perhaps on a connected physical billboard.
The service has a '/draw_shape' endpoint which takes a JSON object
(a Shape TypedDict) describing a shape to draw:
from bottle import HTTPResponse, request, route
from typing import Literal, TypedDict, Union
class Point2D(TypedDict):
x: float
y: float
class Circle(TypedDict):
type: Literal['circle']
center: Point2D # has a nested TypedDict!
radius: float
class Rect(TypedDict):
type: Literal['rect']
x: float
y: float
width: float
height: float
Shape = Union[Circle, Rect] # a Tagged Union / Discriminated Union
@route('/draw_shape')
def draw_shape() -> None:
match request.json: # a Shape?
...
case _:
return HTTPResponse(status=400) # Bad Request
Now, what syntax could we have at the ... inside the "match" statement to
effectively pull apart a Shape?
The current version of PEP 634-636 would require duplicating all the keys
and value types that are defined in Shape's underlying Circle and Rect
types:
match request.json: # a Shape?
case {'type': 'circle', 'center': {'x': float(), 'y':
float()}, \
radius: float()} as circle:
draw_circle(circle) # type is inferred as Circle
case {'type': 'rect', 'x': float(), 'y': float(), \
'width': float(), 'height': float()} as rect:
draw_rect(rect) # type is inferred as Rect
case _:
return HTTPResponse(status=400) # Bad Request
Wouldn't it be nicer if we could use class patterns instead?
match request.json: # a Shape?
case Circle() as circle:
draw_circle(circle)
case Rect() as rect:
draw_rect(rect)
case _:
return HTTPResponse(status=400) # Bad Request
Now that syntax almost works except that Circle and Rect, being TypedDicts,
do not support isinstance() checks. PEP 589 ("TypedDict") did not define how
such isinstance() checks should work initially because it's somewhat complex
to specify. From the PEP:
> In particular, TypedDict type objects cannot be used in isinstance()
tests
> such as isinstance(d, Movie). The reason is that there is no existing
> support for checking types of dictionary item values, since isinstance()
> does not work with many PEP 484 types, including common ones like
List[str].
> [...]
> This is consistent with how isinstance() is not supported for List[str].
Well, what if we (or I) took the time to specify how isinstance() worked
with
TypedDict? Then the match syntax above with TypedDict as a class pattern
would work!
Refining the example above even further, it would be nice if we didn't
have to
enumerate all the different types of Shapes directly in the match-statement.
What if we could match on a Shape directly?
match request.json: # a Shape?
case Shape() as shape:
draw_shape(shape)
case _:
return HTTPResponse(status=400) # Bad Request
Now for that syntax to work it must be possible for an isinstance() check to
work on a Shape, which is defined to be a Union[Circle, Rect], and
isinstance()
checks also aren't currently defined for Union types. So it would be useful
to define isinstance() for Union types as well.
Of course that match-statement is now simple enough to just be rewriten
as an
if-statement:
if isinstance(shape := request.json, Shape):
draw_shape(shape)
else:
return HTTPResponse(status=400) # Bad Request
Now *that* is a wonderfully short bit of parsing code that results in
well-typed objects as output. 🎉
So to summarize, I believe it's possible to support really powerful matching
on JSON objects if we just define how isinstance() should work with a
handful
of new types.
In particular the above example would work if isinstance() was defined for:
* Union[T1, T2], Optional[T]
* T extends TypedDict
* Literal['foo']
For arbitrary JSON beyond the example, we'd also want to support
isinstance() for:
* List[T]
We already support isinstance() for the other JSON primitive types:
* str
* float
* bool
* type(None)
So what do folks think? If I were to start writing a PEP to extend
isinstance()
to cover at least the above cases, would that be welcome?
--
David Foster | Seattle, WA, USA
Contributor to TypedDict support for mypy
I am positing that Python should contain a constant (similar to True,
False, None), called Infinity.
It would be equivalent to `float('inf')`, i.e. a floating point value
representing a non-fininte value. It would be the positive constant;
negative infinity could retrieved via `-Infinity`
Or, to keep float representation the same, the name `inf` could be used,
but that does not fit Python's normal choice for such identifiers (but
indeed, this is what C uses which is the desired behavior of string
conversion)
I think there are a number of good reasons for this constant. For example:
* It is also a fundamental constant (similar to True, False, and None),
and should be representable as such in the language
* Requiring a cast from float to string is messy, and also obviously less
efficient (but this performance difference is likely insignificant)
* Further, having a function call for something that should be a
constant is a code-smell; in general str -> float conversion may throw an
error or anything else and I'd rather not worry about that.
* It would make the useful property that `eval(repr(x)) == x` for
floating point numbers (currently, `NameError: name 'inf' is not defined`)
This makes it difficult to, for example, naively serialize a list of
floats. For example:
```
>>> x = [1, 2, 3, 4]
>>> repr(x)
'[1, 2, 3, 4]'
>>> eval(repr(x)) == x
True
>>> x = [1, 2, 3, float('inf')]
>>> repr(x)
'[1, 2, 3, inf]'
>>> eval(repr(x)) == x
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1, in <module>
NameError: name 'inf' is not defined
```
To me, this is problematic; I would expect it to work seamlessly as it does
with other floating point constants.
A few rebuttals/claims against:
- Creating a new constant (Infinity) which is unassignable may break
existing code
- Converting a float to string is not the same as it is in C. Whil
I also realize that there is `math.inf`, but I argue that the constant is
more fundamental than that, and it still doesn't solve the problem with
`repr()` I described
Thanks,
----
*Cade Brown*
Research Assistant @ ICL (Innovative Computing Laboratory)
Personal Email: brown.cade(a)gmail.com
ICL/College Email: cade(a)utk.edu
```class A:
def __eq__(self, other):
return '__eq__'
class B(A):
def __eq__(self, other):
print(super() == other)
print(super().__eq__(other))
B() == ...```
OUTPUT:
False
__eq__
As you can see here, when you run the code, __uq__ does not get printed twice.
somehow, The expression "super() <OPERATOR> other" does not turn into "super().__OPERATOR__(other)".
this bug is for any operator that you apply on super().
I'd be happy to hear from you why it happens and whether will it be fixed in python3.10.
Regards - Jonatan.
From PEP 636 (Structural Pattern Matching):
> Mapping patterns: {"bandwidth": b, "latency": l} captures the
"bandwidth" and "latency" values from a dict. Unlike sequence patterns,
extra keys are ignored.
It surprises me that ignoring extra keys would be the *default*
behavior. This seems unsafe. Extra keys I would think would be best
treated as suspicious by default.
* Ignoring extra keys loses data silently. In the current proposal:
point = {'x': 1, 'y': 2, 'z': 3)
match point:
case {'x': x, 'y': y}: # MATCHES, losing z O_O
pass
case {'x': x, 'y': y, 'z': z}: # will never match O_O
pass
* Ignoring extra keys is inconsistent with the handling of sequences: We
don't allow extra items when using a destructuring assignment to a sequence:
p = [1, 2]
[x, y] = p
[x, y, z] = p # ERROR: ValueError: not enough values to unpack
(expected 3, got 2) :)
* Ignoring extra keys in mapping patterns is inconsistent with the
current proposal for how sequence patterns match data:
point = [1, 2, 3]
match point:
case [x, y]: # notices extra value and does NOT match :)
pass
case [x, y, z]: # matches :)
pass
* Ignoring extra keys is inconsistent with TypedDict's default "total"
matching behavior:
from typing import TypedDict
class Point2D(TypedDict):
x: int
y: int
p1: Point2D = {'x': 1, 'y': 2}
p2: Point2D = {'x': 1, 'y': 2, 'z': 3) # ERROR: Extra key 'z' for
TypedDict "Point2D" :)
* It is *possible* to force an exact key match with a pattern guard but
it's clumsy to do so.
It should not be clumsy to parse strictly.
point = {'x': 1, 'y': 2, 'z': 3)
match point:
# notices extra value and does NOT match, but requires ugly
guard :/
case {'x': x, 'y': y, **rest} if rest == {}:
pass
case {'x': x, 'y': y, 'z': z, **rest} if rest == {}:
pass
To avoid the above problems, **I'd advocate for disallowing extra keys
in mapping patterns by default**. For cases where extra keys want to be
specifically allowed and ignored, I propose allowing a **_ wildcard.
Some examples that illustrate behavior when *disallowing* extra keys in
mapping patterns:
1. Strict parsing
from typing import TypedDict, Union
Point2D = TypedDict('Point2D', {'x': int, 'y': int})
Point3D = TypedDict('Point3D', {'x': int, 'y': int, 'z': int})
def parse_point(point_json: dict) -> Union[Point2D, Point3D]:
match point_json:
case {'x': int(x), 'y': int(y)}:
return Point2D({'x': x, 'y': y})
case {'x': int(x), 'y': int(y), 'z': int(z)}:
return Point3D({'x': x, 'y': y, 'z': z})
case _:
raise ValueError(f'not a valid point: {point_json!r}')
2. Loose parsing, discarding unknown data.
Common when reading JSON-like data when it's not necessary to output
it again later.
from typing import TypedDict
TodoItem_ReadOnly = TypedDict('TodoItem_ReadOnly', {'title': str,
'completed': bool})
def parse_todo_item(todo_item_json: Dict) -> TodoItem_ReadOnly:
match todo_item_json:
case {'title': str(title), 'completed': bool(completed), **_}:
return TodoItem_ReadOnly({'title': title, 'completed':
completed})
case _:
raise ValueError()
input = {'title': 'Buy groceries', 'completed': True,
'assigned_to': ['me']}
print(parse_todo_item(input)) # prints: {'title': 'Buy groceries',
'completed': True}
3. Loose parsing, preserving unknown data.
Common when parsing JSON-like data when it needs to be round-tripped
and output again later.
from typing import Any, Dict, TypedDict
TodoItem_ReadWrite = TypedDict('TodoItem_ReadWrite', {'title': str,
'completed': bool, 'extra': Dict[str, Any]})
def parse_todo_item(todo_item_json: Dict) -> TodoItem_ReadWrite:
match todo_item_json:
case {'title': str(title), 'completed': bool(completed),
**extra}:
return TodoItem_ReadWrite({'title': title, 'completed':
completed, 'extra': extra})
case _:
raise ValueError()
def format_todo_item(item: TodoItem_ReadWrite) -> Dict:
return {'title': item['title'], 'completed': item['completed'],
**item['extra']}
input = {'title': 'Buy groceries', 'completed': True,
'assigned_to': ['me']}
output = format_todo_item(parse_todo_item(input))
print(output) # prints: {'title': 'Buy groceries', 'completed':
True, 'assigned_to': ['me']}
Comments?
--
David Foster | Seattle, WA, USA
Contributor to TypedDict support for mypy
Greetings all,
We can do one type annotations like list[int] and dict[str, int], my question: why don’t we have a similar thing in matching patterns? Especially if you want to enforce a type on a sequence or a dictionary with unknown length.
For example,
ls = [3.14, ‘hello’, 1, 2, 3, 4, ...]
d = {‘name’: 1, ‘age’: 25, ...}
match ls:
case [float(), str(), *rest] if all(isinstance(elem, int) for elem in rest): ... (do something with rest)
We do the same thing with the following:
case [float(), str(), *int() as rest]: (do something with rest)
match d:
case **rest if all(isinstance(k, str) and isinstance(v, int) for k, v in rest.items()): (do something with rest)
We do the same thing with the following:
case **{str() as keys: int() as values} as rest: (do something with rest or with keys (=rest.keys()) and values (=rest.values()) directly)
{str() as k: int() as v} matches a dictionary that has at least one structure of str(): int() but what would the values of k and v be in this case? Is It the first match since python dictionaries are now ordered? Or this is not permitted in dictionary matching? I saw examples like {‘string literal’: int() as v} on the web.
What do you think of this idea?
Abdulla
