Date: Fri, 26 Jun 2020 18:47:44 +0200
From: Hans Ginzel <hans(a)matfyz.cz>
To: Hans Ginzel <hans(a)artax.karlin.mff.cuni.cz>
Subject: Access (ordered) dict by index; insert slice
Hello,
thank you for making dict ordered.
Is it planned to access key,value pair(s) by index? See https://stackoverflow.com/a/44687752/2556118 for example. Both for reading and (re)writing?
Is it planned to insert pair(s) on exact index? Or generally to slice? See splice() in Perl, https://perldoc.perl.org/functions/splice.html.
Use case: Represent database table metadata (columns). It is useful as to access columns both by name and by index as to insert column on specific position, https://dev.mysql.com/doc/refman/8.0/en/alter-table.html, “ALTER TABLE ADD COLUMN [FIRST |AFTER col]” (consider default order or table storage size optimisation by aligning).
Thank you in advance,
Hans
PS1: Named tuples cannot be used, are immutable.
PS2: See https://metacpan.org/pod/perlref#Pseudo-hashes:-Using-an-array-as-a-hash
Hello!
I'm not sure I'm addressing the right audience here, so please direct me
to the appropriate channel if that's the case...
My name is Andras Tantos and I'm working on a Python library desscribing
HW designs. I came across this problem of __getitem__ and co. not
supporting kwargs. Apparently this extension was proposed and rejected
as PEP 472.
Apart from my use-case, which is arguably a corner-case and not worth
modifying the language for, I believe there are two important use-cases
that are worth considering with the latest improvements in the language:
1. With the recent type-hint support, the feature could be made way more
descriptive if this PEP got implemented.
For example, instead of doing the following:
def func(in: Dict[str, int])
one could write:
def func(in: Dict[key=str, value=int])
2. It would also make 'generic classes' much cleaner to implement,
similar to the way type-hints look. Consider the following code:
class_Generic(object):
Specializations = []
@classmethod
def__getitem__(cls, *args):
name = f"Generic_{len(cls.Specializations)}"
Specialized = type(name, (cls,), {"specials": tuple(args)})
cls.Specializations.append(Specialized)
returnSpecialized
def__init__(self, value= None):
self.value = value
def__str__(self):
ifhasattr(self, "specials"):
return(f"[{type(self)}- "+ ",".join(str(special) forspecial
inself.specials) + f"] - {self.value}")
else:
return(f"[{type(self)}- GENERIC"+ f"] - {self.value}")
Generic = _Generic()
#g = Generic() - fails because of no specialization is given
s1 = Generic[12]()
s2 = Generic[42]("Hi!")
print(s1)
print(s2)
Running this simple example results in:
python3 -i python_test.py
[<class '__main__.Generic_0'> - 12] - None
[<class '__main__.Generic_1'> - 42] - Hi!
You can see how the specialized parameters got passed as well as the
ones to '__init__'. Obviously, in real code the idea would be to
filter generic parameters and set up 'Specialized' with the right
set of methods and arguments.
Now, without kwargs support for __getitem__, it's impossible to pass
named arguments to the specialization list, which greatly limits the
usability of this notation.
I don't know how convincing these arguments and use-cases are for you,
but could you advise me about how to start the 'ball rolling' to drum-up
support for re-activating this PEP?
Thanks again,
Andras Tantos
Even after years of coding it sometimes takes me a moment to correctly
parse expressions like `min(max(value, minimum), maximum)`, especially
when the parentheses enclose some local computation instead of only
references, and the fact that `min` actually needs the *maximum* value
as an argument (and vice-versa) certainly doesn't help.
It's a little surprising to me how shorthand functions akin to CSS's
`clamp()` aren't more popular among modern programming languages. Such a
tool is, in my opinion, even more missed in Python, what with it being
focussed on readability and all. There would likely also be some
(probably minor) performance advantages with implementing it at a
builtins level.
Example usage:
>>> val = 100
>>> clamp(10, val, 50)
50
>>> val = 3
>>> clamp(10, val, 50)
10
>>> val = 25
>>> clamp(10, val, 50)
25
I'm undecided whether I would like `clamp`, `minmax` or something else
as a name. I'm curious of other ideas.
As far as the signature is concerned, I like both `clamp(min, val, max)`
for the logical position of the three arguments (which mirrors
expressions such as `min < val < max`) and `clamp(val, min=x, max=y)`. I
prefer the former, but declaring them as normal positional-and-keyword
arguments would allow the programmer to use an alternative order if they
so choose.
CPython at the very least has 2 different type of native states: Interpreter & Module state.
Unfortunately, the multi-phase initialization has a weakness when it comes to Module states.
You can't access the module state without a pointer to the module.
PyState_GetModule from a standpoint looks to be the obvious answer to use, but it's documentation states it's unfit for multi-phase initialization.
I'm proposing an idea here for discussion on a new state system for at least CPython.
Tier 1: Core state
- This state lives within CPython's core binary and exists the entire lifetime of the binary.
- The data held within this state is available to the main interpreter and subsequent sub-interpreters. (Example: sys.executable)
Tier 2-1: Interpreter state (Branches from Core state)
- This state lives within CPython's core binary and is tied to a specific interpreter.
- The data held within this state is only available to the interpreter it's tied to. (Example: Modules loaded into memory)
Tier 2-2: Extension state (Branches from Core state)
- This state lives within any CPython's core binary EXCEPT it's size and structure is defined by the extension CPython has loaded.
- The purpose of this state is to allow an extension to hold data that can't be tied to a specific module. (Examples can be: Windows WSA, MySQL)
Tier 3-1: Thread state (Branches from Interpreter State)
- This state lives within CPython's core binary and is tied to a specific Python thread (IE: Threading library threads),
- The data held within this state is only available to the thread it's tied to. (No known examples available)
Tier 3-2: Module state (Branches from Interpreter State)
- This type of state is already available in CPython, explaining it is not required.
Hi All
This is related to discussion
https://mail.python.org/archives/list/python-ideas@python.org/thread/KWAOQF…
In Python, lists don't have a join method. Instead, it's strings that have
the join method. Hence we have:
>>> ', '.join('abcde')
'a, b, c, d, e'
The intermediate method we can save and store and use again.
>>> joiner = ', '.join
>>> joiner('fghijk')
'f, g, h, i, j, k'
We can do something similar when clamping, clipping or trimming a value.
For example, suppose we want limits on the room temperature, that the
thermostat cannot override.
>>> aircon_clipper = Clamper(50, 80)
>>> thermostat_temp = 40
>>> target_temp = aircon_temp_clipper(thermostat_temp)
>>> target_temp
50
What I like best about this is that the name of the function --
aircon_temp_clipper -- gives the reason for the clipping (or clamping). And
this name can be chosen to suit the domain. I also like that it wraps the
high and low values (50 and 80) as data used by a method.
Here's an implementation of Clamper.
class Clamper:
def __init__(self, lo, hi):
if not lo <= hi:
raise ValueError
self._lo = lo
self._hi = hi
def __call__(self, num):
lo, hi = self._lo, self._hi
at_least_lo = (lo <= num)
at_most_hi = (hi >= num)
if at_least_lo and at_most_hi:
return num
elif at_least_lo:
return hi
elif at_most_hi:
return lo
else:
raise ValueError
If performance is important to you, you could instead use a closure to
create a function. This would also allow you to provide a docstring.
I hope this helps at least some of the people some of the time.
--
Jonathan
Is there any reason why collections.abc.[Sequence|MutableSequence] do
not have "__eq__" for free?
I was writing some tests now and was caught by surprise there.
(chcking the docs, if my custom MutbaleSequence also
inherits from "collections.abc.Set" I will have working "__eq__" - and
"__ne__", methods
but also weirdly behaved "__le__" and cousins as a side efffect - weird
enough
I think i will just hand-write the "__eq__")
While theoretically adding "__eq__" and "__ne__" _could_ hab
backward compatibility issues, I think it is more probable it would
actually fix some undetected bugs i a couple projets, as the default __eq__
is
an identity comparison (is).
Anyway, maybe there is a reason it is not a given. Any thoughts?
With lib2to3 going away (https://bugs.python.org/issue40360), it seems to
me that some of its functionality for handling "whitespace" can be fairly
easily added to the ast module. (By "whitespace", I mean things like
indent, dedent, comments, backslash; and also the ability to manipulate the
encoded bytes in the original source.)
Off the top of my head, I know of the following projects that use lib2to3
or similar to access the "whitespace" in the parse tree and will need a new
solution:
yapf, black, mypy, pytype, kythe.
(If they don't use lib2to3, they need to maintain a custom parser that
changes with each release of Python, so my proposal would potentially help
them.)
Are there other projects that need access to the parse tree and
"whitespace"?
I propose implementing an optional pass over the parse tree that records
lib2to3's "prefix" with each leaf node. The interface would be something
like:
def detect_encoding(source: bytes) -> str
def parse_with_whitespace(source: bytes, encoding: str, filename: str) ->
ast.Module
def unparse_bytes(tree: ast.Module) -> bytes
def unparse_str(tree: ast.Module) -> str
# Various convenience functions/properties, similar to
pytree.next_sibling etc.
parse_with_whitespace() calls ast.parse(), then does a pass over the parse
tree, adding to the leaf nodes:
prefix: str # whitespace and comments preceding token in the input
pieces: List # see below
col_byte_offset: int # start byte offset within line
src_byte_offset: int # start byte offset within source
The "pieces" field is intended to handle things like:
x = 'abc' \
"def"
y = (f'abc{x}' # comment
"def")
Each "piece" would include:
- byte offset from the beginning of the token (negative for a prefix
piece)
- detailed type (single-quote string, double-quote string, format-string,
int, float, etc.)
- source bytes
- decoded value (Unicode str)
The ast.Module class would also be extended with additional attributes that
apply to the entire source, such as the encoding.
All of this is quite fiddly, but I already have some code for dealing with
the conversions between byte and string offsets, so I don't anticipate a
huge amount of work. (The design is also inherently a bit inefficient; but
I don't want to get involved with the internals of compile().)
A related item: "Parser module in the stdlib":
https://mail.python.org/archives/list/python-dev@python.org/thread/RHZ6JOEX…
Why not to allow tuple as a list index?
>>> T = [[11, 12, 5, 2], [15, 6, 10], [10, 8, 12, 5], [12, 15, 8, 6]]
>>> print(T[1][2])
10
>>> print(T[1, 2])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: list indices must be integers or slices, not tuple
https://stackoverflow.com/questions/40361743/python-using-tuples-as-list-in…
The official doc of the partial function from the functools stdlib module says it "is used for partial function application which “freezes” some portion of a function’s arguments and/or keywords resulting in a new object with a simplified signature".
However, whereas positional arguments are effectively "frozen" (cannot be altered in the partial object), keyword arguments can in fact not be frozen as they always will be overridden by the ones passed to the partial object.
The "and/or keywords" part in the quote above thus sounds wrong when applied to the current implementation, and is more accurately describing the following situation :
def partial(func, /, *args, **keywords):
def newfunc(*fargs, **fkeywords):
return func(*args, *fargs, **keywords, **fkeywords) # instead of the current func(*args, *fargs, **{**keywords, **fkeywords}) value
newfunc.func = func
newfunc.args = args
newfunc.keywords = keywords
return newfunc
Shouldn't we provide the above implementation - which really freezes the given positional AND keyword arguments - in a new partial function to be added to the functools module, and fix the docs accordingly ?
On the signature side, we would also actually always get a "new object with a simplified signature" as stated in the doc, which is not the case currently.
If we consider the following code :
import functools
def F(arg, opt = 'BaseValue') :
return (arg, opt)
G = functools.partial(F, opt = 'NewValue')
the partial function G's signature is (arg, opt = 'NewValue') and is not really simplified compared to the initial (arg, opt = 'BaseValue') one.
The only effect of partial on the function signature is to alter the default values of the specified keyword arguments.
With the proposed implementation, the signature would actually be simplified to (arg).
Sorry, this got lost in the shuffle without being sent, and the
conversation seems to have died down. but since I wrote it ...
On Sat, Jul 4, 2020 at 8:25 AM Stephen J. Turnbull <
turnbull.stephen.fw(a)u.tsukuba.ac.jp> wrote:
> Christopher Barker writes:
>
> > I am really confused here -- I'm suggesting that is should NOT be
> possible
> > to tell the difference.
>
> But it is possible to tell the difference, so that suggestion is moot.
I meant in the context of putting things in, or testing whether they are
in, sets. Not in any other context.
There are languages where there is no numeric tower, just numbers, and
> [0][0], [0][0.0], and [0][0+0j] all just work. Python isn't one of
> them. Fewer places you can tell in Python 3 than in Python 2, I
> believe, but several remain.
Sure, but putting them in sets, or using them as a key in a dict is not one
of them.
set.intern does yield the same result regardless of what's passed in:
> whichever is in the set. That's *why* I can call it "intern".[1]
Fair enough. You can of course propose any functionality you want. I’m
suggesting that my understanding of what you are proposing doesn’t match
well with the rest of how sets work.
But to see if I understand correctly, you couldn’t quite use a set to
intern objects: once 2.0 was in there you could not put 2 in as well. You
could know that 2.0 was already in there when you tried to add 2, but it
would still never be added.
It seems if you really want to intern objects, you’d need something kind of
like, but not quite, a set().
Note: this all is only an issue for object that are of a different type,
but can compare equal and have the same hash - maybe the numeric types are
the only ones in the stdlib for which this is the case. Nevertheless, they
are commonly used :-)
Nope: turns out NamedTuples and Tuple are the "same" as far as sets are
concerned as well.
In [66]: s = {p}
In [67]: s
Out[67]: {Point(x=2, y=3)}
In [68]: s.add((2,3))
In [69]: s
Out[69]: {Point(x=2, y=3)}
In [70]: s = {(2,3)}
In [71]: s.add(p)
In [72]: s
Out[72]: {(2, 3)}
So if you want something that treats different types with the same value as
different, you really will need a different storage mechanism than a set.
-CHB
