On Sun, Jan 16, 2022 at 7:35 PM Steven D'Aprano wrote:

How does this work for you?

f{1, 2, 3}

While it's tempting, it does create an awkward distinction. f(1, 2, 3) # look up f, call it with parameters f[1, 2, 3] # look up f, subscript it with paramters f{1, 2, 3} # construct a frozenset And that means it's going to be a bug magnet. Are we able to instead make a sort of vector literal? <1, 2, 3> Unfortunately there aren't many symbols available, and Python's kinda locked into a habit of using just one at each end (rather than, say, (<1, 2, 3>) or something), so choices are quite limited. ChrisA

On Sun, Jan 16, 2022 at 11:18 PM Steven D'Aprano wrote:

On Sun, Jan 16, 2022 at 09:18:40PM +1100, Chris Angelico wrote:

...

While it's tempting, it does create an awkward distinction.

f(1, 2, 3) # look up f, call it with parameters f[1, 2, 3] # look up f, subscript it with paramters f{1, 2, 3} # construct a frozenset

You forgot

f"1, 2, {x+1}" # eval some code and construct a string

Not to mention:

r(1, 2, 3) # look up r, call it with parameters r[1, 2, 3] # look up r, subscript it r"1, 2, 3" # a string literal

Strings behave differently in many many ways. Are there any non-string types that differ?

Reading this makes my eyes bleed:

>>> <1, 2, 3> < <1, 2, 3, 4> True

Fair point, but I can't imagine people comparing two literals like that. It's not quite as bad if you replace the left side with a variable or calculation, though it's still kinda weird.

...

Unfortunately there aren't many symbols available, and Python's kinda locked into a habit of using just one at each end (rather than, say, (<1, 2, 3>) or something), so choices are quite limited.

Triple quoted strings say hello :-)

See above, strings are different, and people treat them differently.

{{1, 2, 3}} would work, since that's currently a runtime error. But I prefer the f{} syntax.

Yeah, I think that ship has sailed. Double punctuation just isn't Python's thing, so there aren't really any good ways to shoehorn more data types into fewer symbols. ChrisA

On 2022-01-16 08:27, Steven D'Aprano wrote: [snip]

...

...

...

dis.dis("frozenset({1, 2, 3})") 1 0 LOAD_NAME 0 (frozenset) 2 BUILD_SET 0 4 LOAD_CONST 0 (frozenset({1, 2, 3})) 6 SET_UPDATE 1 8 CALL_FUNCTION 1 10 RETURN_VALUE

Got that? To create a frozenset of literals, first the compiler creates a frozenset constant containing what you wanted. Then at runtime, it:

- looks up frozenset in globals and builtins; - loads the pre-prepared frozenset (which is exactly what we want); - creates a new set from that frozenset; - calls the frozenset() function on that set to create a new frozenset that duplicates the pre-prepared one; - and finally garbage-collects the temporary set.

[snip] Not quite as bad as that:

...

...

f = frozenset({1, 2, 3}) f is frozenset(f) True

Summary: Further information is provided, which suggests that it may be best to amend Python so that "frozenset({1, 2, 3})" is the literal for eval("frozenset({1, 2, 3})"). Steve D'Aprano correctly notes that the bytecode generated by the expression x in {1, 2 ,3} is apparently not optimal. He then argues that introducing a frozenset literal would allow x in f{1, 2, 3} # New syntax, giving a frozenset literal would allow better bytecode to be generated. However, the following has the same semantics as "x in {1, 2, 3}" and perhaps gives optimal bytecode.

...

...

import dis dis.dis("x in (1, 2, 3)") 1 0 LOAD_NAME 0 (x) 2 LOAD_CONST 3 ((1, 2, 3)) 4 COMPARE_OP 6 (in) 6 RETURN_VALUE

For comparison, here's the bytecode Steve correctly notes is apparently not optimal.

...

...

dis.dis("x in {1, 2, 3}") 1 0 LOAD_NAME 0 (x) 2 LOAD_CONST 3 (frozenset({1, 2, 3})) 4 COMPARE_OP 6 (in) 6 RETURN_VALUE

Steve states that "x in {1, 2, 3}" when executed calls "frozenset({1, 2, 3})", and in particular looks up "frozenset" in builtins and literals. I can see why he says that, but I've done an experiment that suggests otherwise.

...

...

bc = compile("x in {1, 2, 3}", "filename", "eval") eval(bc, dict(x=1)) True eval(bc, dict(x=1, frozenset="dne")) True

I suspect that if you look up in the C-source for Python, you'll find that dis.dis ends up using frozenset({1, 2, 3}) as the literal for representing the result of evaluating frozenset({1, 2, 3}). The following is evidence for this hypothesis:

...

...

frozenset({1, 2, 3}) frozenset({1, 2, 3}) set({1, 2, 3}) {1, 2, 3} set([1, 2, 3]) {1, 2, 3} set([]) set()

To conclude, I find it plausible that: 1. The bytecode generated by "x in {1, 2, 3}" is already optimal. 2. Python already uses "frozenset({1, 2, 3})" as the literal representation of a frozenset. Steve in his original post mentioned the issue https://bugs.python.org/issue46393, authored by Terry Reedy. Steve rightly comments on that issue that "may have been shadowed, or builtins monkey-patched, so we cannot know what frozenset({1, 2, 3}) will return until runtime." Steve's quite right about this shadowing problem. In light of my plausible conclusions I suggest his goal of a frozenset literal might be better achieved by making 'frozenset' a keyword, much as None and True and False are already keywords.

...

...

True = False File "<stdin>", line 1 SyntaxError: can't assign to keyword

Once this is done we can then use frozenset({1, 2, 3}) as the literal for a frozenset, not only in dis.dis and repr and elsewhere, but also in source code. As a rough suggestion, something like from __future__ import literal_constructors_as_keywords would prevent monkey-patching of set, frozenset, int and so forth (just as True cannot be monkeypatched). I thank Steve for bringing this interesting question to our attention, for his earlier work on the issue, and for sharing his current thoughts on this matter. It's also worth looking at the message for Gregory Smith that Steve referenced in his original post. https://mail.python.org/pipermail/python-ideas/2018-July/051902.html Gregory wrote: frozenset is not the only base type that lacks a literals leading to loading values into these types involving creation of an intermediate throwaway object: bytearray. bytearray(b'short lived bytes object') I hope this helps. -- Jonathan

On Mon, Jan 17, 2022 at 9:55 AM Steven D'Aprano wrote:

On Sun, Jan 16, 2022 at 04:50:40PM +0000, MRAB wrote:

...

Not quite as bad as that:

...

...

...

f = frozenset({1, 2, 3}) f is frozenset(f) True

Mark suggested that on the bug tracker too, but that's not relevant. As I replied there:

...

...

...

def f(): ... return frozenset({1, 2, 3}) ... a = f.__code__.co_consts[1] a frozenset({1, 2, 3}) b = f() assert a == b a is b False

Each time you call the function, you get a distinct frozenset object.

And semantically, I believe that's correct. The set display MUST create a new set every time, and since the compiler/optimizer can't assume that the name 'frozenset' hasn't been rebound, it has to then call the function to get its result. So it will, by language definition, build a set from the given values, then build a frozenset from that. It optimizes "add 1, add 2, add 3" down to "here's a frozenset, add them", but it can't skip the whole job. def frozenset(stuff): print("You can't optimize me out") stuff.remove(2) return builtins.frozenset(stuff) If the compiler did any of the optimizations it's currently not doing, this code would break. The only way to solve this is an actual frozenset literal, and I would love to have one, but it stands or falls on the syntax. ChrisA

On Sun, Jan 16, 2022 at 11:41:52PM +1100, Chris Angelico wrote:

On Sun, Jan 16, 2022 at 11:18 PM Steven D'Aprano wrote:

...

Not to mention:

r(1, 2, 3) # look up r, call it with parameters r[1, 2, 3] # look up r, subscript it r"1, 2, 3" # a string literal

Strings behave differently in many many ways. Are there any non-string types that differ?

There are plenty of non-string types which differ :-) Differ in what way? I don't understand your question. You were concerned that adding a prefix to a delimiter in the form of f{...} would be a bug magnet, but we have had prefixes on delimiters for 30 years in the form of r"..." etc, and it hasn't been a problem. I mean, sure, the occasional beginner might get confused and write len{mystring} and if by some fluke they call f() rather than len() they will get a silent failure instead of a SyntaxError, but is this really a serious problem that is common enough to get labelled "a bug magnet"? I've been coding in Python for two decades and I still occassionally mess up round and square brackets, especially late at night, and I won't tell you how often I write my dict displays with equal signs {key=value}, or misspell str.center. And I still cringe about the time a few years back where my brain forgot that Python spells it "None" rather than "nil" like in Pascal, and I spent about an hour writing a ton of "if obj is nil"... tests. Typos and brain farts happen.

...

Reading this makes my eyes bleed:

>>> <1, 2, 3> < <1, 2, 3, 4> True

Fair point, but I can't imagine people comparing two literals like that.

They don't have to be literals inside the brackets. Especially in the REPL, `{*a} < {*b}` is a quick way of testing that every element of a is an element of b. [...]

...

Triple quoted strings say hello :-)

See above, strings are different, and people treat them differently.

Do they? How are they different? You have a start delimiter and an end delimiter. The only difference I see is that with strings the delimiter is the same, instead of a distinct open and close delimiter. But that difference is surely not a reason to reject the use of a prefix. "We can't use a prefix because the closing delimiter is different from the opening delimiter" does not follow. -- Steve

On 2022-01-16 16:11, Steven D'Aprano wrote:

Do they? How are they different? You have a start delimiter and an end delimiter.

The only difference I see is that with strings the delimiter is the same, instead of a distinct open and close delimiter. But that difference is surely not a reason to reject the use of a prefix.

Well, there is a big difference, which is that the stuff between other delimiters (parentheses, brackets, etc.) is wholly constrained by Python syntax, whereas the stuff between string delimiters is free-form text, with only a few restrictions (like not being able to use the delimiter itself, or to include newlines in single-quoted strings). Whether that difference is important for your proposal I won't address right now. But it is a big difference. It also greatly affects how people view the code, since syntax highlighters will often color an entire string literal with the same color, whereas they don't typically do that for other kinds of delimited chunks, instead highlighting only the delimiters themselves. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown

On Mon, 17 Jan 2022 at 00:46, Christopher Barker wrote:

I’m a bit confused — would adding a “literal” form for frozenset provide much, if any, of an optimization? If not, that means it’s only worth doing for convenience.

How often do folks need a frozen set literal? I don’t think I’ve ever used one.

You won't have used one because they have not yet existed (hence this thread).

If we did, then f{‘this’: ‘that’} should make a frozen dict, yes?

A frozen dict would also be useful but the implementation doesn't exist. If it did exist then in combination with this proposal that syntax for frozen dicts would be an obvious extension. A more relevant question right now is if any other set syntax should apply to frozensets e.g. should this work: >>> squares = f{x**2 for x in range(10)} -- Oscar

On Mon, Jan 17, 2022 at 9:58 AM Oscar Benjamin wrote:

On Mon, 17 Jan 2022 at 00:46, Christopher Barker wrote:

...

I’m a bit confused — would adding a “literal” form for frozenset provide much, if any, of an optimization? If not, that means it’s only worth doing for convenience.

How often do folks need a frozen set literal? I don’t think I’ve ever used one.

You won't have used one because they have not yet existed (hence this thread).

Although we don't have frozenset literal now, we have frozenset. So we can estimate how frozenset literal is useful by seeing how frozenset is used. Unless how the literal improve codes is demonstrated, I am -0.5 on new literal only for consistency. Regards, -- Inada Naoki

On Sun, Jan 16, 2022 at 04:23:47PM -0800, Brendan Barnwell wrote:

On 2022-01-16 16:11, Steven D'Aprano wrote:

...

Do they? How are they different? You have a start delimiter and an end delimiter.

The only difference I see is that with strings the delimiter is the same, instead of a distinct open and close delimiter. But that difference is surely not a reason to reject the use of a prefix.

Well, there is a big difference, which is that the stuff between other delimiters (parentheses, brackets, etc.) is wholly constrained by Python syntax, whereas the stuff between string delimiters is free-form text, with only a few restrictions (like not being able to use the delimiter itself, or to include newlines in single-quoted strings).

Which is also wholly constrained by Python syntax, seeing as they are Python strings :-)

Whether that difference is important for your proposal I won't address right now. But it is a big difference. It also greatly affects how people view the code, since syntax highlighters will often color an entire string literal with the same color, whereas they don't typically do that for other kinds of delimited chunks, instead highlighting only the delimiters themselves.

Are any of these differences relevant to putting a prefix on the opening delimiter? If they are not, why mention them? We could also talk about the difference between the numeric value of the ASCII symbols, or the number of pixels in a " versus a { glyph, or the linguistic history of the words "quotation mark" versus "curly bracket" too, but none of these things seem to be any more relevant than whether IDEs and syntax colourizers colour "..." differently to {1, None, 4.5}. Can we bypass what could end up being a long and painful discussion if I acknowledge that, yes, frozensets are different to strings, and so the syntax is different. (Different things necessarily have different syntax. Otherwise they would be indistinguishable.) - Sets (like lists, tuples and dicts) are compound objects that contain other objects, so their displays involve comma-separated items; - string (and byte) literals are not, except in the sense that strings can be considered to be an array of single-character substrings. I thought that was so obvious and so obviously irrelevant that it didn't even need mentioning. Perhaps I am wrong. (It has to happen eventually *wink*) If somebody can explain *why* that matters, rather than just declare that it rules out using a prefix, I would appreciate the education. Hell, even if your argument is just "Nope, I just don't like the look of it!", I would respect that even if I disagree. Aesthetics are important, even when they are totally subjective. If it helps, Julia supports this syntax for typed dicts: Dict{keytype, valuetype}(key => value) where the braces {keytype, valuetype} are optional. That's not a display syntax as such, or a prefix, but it is visually kinda similar. Here are some similar syntax forms with prefixes: * Dylan list displays: #(a, b, c) * Smalltalk drops the comma separators: #(a b c) * Scheme and Common Lisp: '(a b c) and double delimiters: * Pike: ({ a, b, c }) * Pike dicts: ([ a:b, c:d ]) Not that we could use any of those as-given. -- Steve

On 2022-01-16 00:27, Steven D'Aprano wrote:

It seems to me that all of the machinery to make this work already exists. The compiler already knows how to create frozensets at compile-time, avoiding the need to lookup and call the frozenset() builtin. All we need is syntax for a frozenset display.

How does this work for you?

f{1, 2, 3}

I don't like that syntax. In the first place, as others have noted, it treads too close to existing function-call and indexing syntax. In those syntaxes, `f` is a name, whereas here it is not a name but just a syntactic marker (as in f-strings). In the second place, I don't like the idea of using prefixes to change the types of objects like this. As far as I know we only have one example of that, namely byte strings. The other string prefixes do not create different types, they just affect how the literal is eventually parsed into a string. I would prefer some sort of augmented bracket notation, like {:1, 2, 3:} or something along those lines. I do think it would be nice to have a frozenset literal, because I'm a purity-beats-practicality person and it bugs me that there isn't one. However, from that perspective I'd much rather have a frozen-dict type and then a syntax for it. :-) -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown

On Sun, Jan 16, 2022 at 3:14 PM Paul Moore wrote:

I may just be reiterating your point here (if I am, I'm sorry - I'm not completely sure), but isn't that required by the definition of the frozenset function. You're calling frozenset(), which is defined to "Return a new frozenset object, optionally with elements taken from iterable". The iterable is the (non-frozen) set {1, 2, 3}.

Exactly. In a way, what this might do is open the door to interning (some) frozen sets, like cPython does for some ints and strings. But that's only helpful is they are very heavily used, which they are not. And I'm sure a way to intern them could be done without a literal anyway. (using f{...} as a frozenset literal) does something different - it

returns the *same* object, compiled once at function definition time, every time it's called.

why/how would it do that? It *could* do that -- as above, with interning. but: def fun(): return "some string" doesn't return the same string, unless it's iterned, which is an implementation detail, yes? Stephen -- I'm going to paraphrase you now: We don't make changes to Python syntax unless there is a compelling reason. I'm really lost on what the compelling reason is for this one? There are any number of python types with no "literal" (well, not any number, it's quite defined. but still) heck, we don't even have literals for Decimal. Why this one? -CHB --- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

On Mon, Jan 17, 2022 at 1:14 PM Christopher Barker wrote:

why/how would it do that? It *could* do that -- as above, with interning. but:

def fun(): return "some string"

doesn't return the same string, unless it's iterned, which is an implementation detail, yes?

Not sure what you mean. That's a constant, so it'll always return the exact same object, surely? ChrisA

On Mon, Jan 17, 2022 at 10:15:50AM +0900, Inada Naoki wrote:

Unless how the literal improve codes is demonstrated, I am -0.5 on new literal only for consistency.

I never suggested adding this "for consistency". -- Steve

On Sun, Jan 16, 2022 at 6:34 PM Chris Angelico wrote:

...

def fun(): return "some string"

doesn't return the same string, unless it's iterned, which is an implementation detail, yes?

Not sure what you mean. That's a constant, so it'll always return the exact same object, surely?

I *think* that's only if it's interned -- and in any case, is a guarantee of the language, or an optimization? I tried to test with a longer string, and it was the same one, but then I found in this arbitrary post on the internet: ... in Python 3.7, this has been changed to 4096 characters ( I guess I haven't played with that since 3.7) -- I haven't actually tried with a string literal linger than 4096 chars :-) But this certainly doesn't: In [1]: def fun(): ...: return [1,2,3] ...: In [2]: l1 = fun() In [3]: l2 = fun() In [4]: l1 is l2 Out[4]: False So the issue is immutability and interning, not "literal display". My point is that a frozenset litteral could open the door to interning frozen sets, but that could probably be done anyway. And why? Are they heavily used in any code base? -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

On Mon, Jan 17, 2022 at 2:01 PM Steven D'Aprano wrote:

...

Yes. ONLY on strings. That's exactly what I said. Strings are different. For starters, we already have multiple different data types that can come from quoted literals, plus a non-literal form that people treat like a literal (f-strings). Is there any non-string type that doesn't follow that pattern?

So... not "ONLY" (your emphasis, not mine) strings. As you say, there are already two different data types, strings and bytes, plus a non-literal executable code (f-strings).

Again, I don't understand your point. You have just told us that there are already non-string literals and types that use a prefix on the delimiter, and then you ask me if there are any non-string types that follows that pattern. Haven't you just answered your own question?

An f-string still yields a string. It's not a literal but it's still a string. A bytestring is still a string. It's not a Unicode string but it's still a string. These are not the same as lists, tuples, sets, dicts, etc, which contain arbitrary objects. ONLY strings. You just picked up on the part where I said "not only literals" and got it completely backwards. Strings are not the same as lists. Strings are not the same as tuples. Strings are the only data type in Python that has prefixes that determine the data type you get. Strings are TREATED DIFFERENTLY by programmers, which is why f-strings get treated like string literals. Strings. Are. Different. I do not know how to make this any clearer. If you do not understand my position, please stop misrepresenting me. ChrisA

On Mon, Jan 17, 2022 at 4:38 PM Christopher Barker wrote:

On Sun, Jan 16, 2022 at 6:34 PM Chris Angelico wrote:

...

...

def fun(): return "some string"

doesn't return the same string, unless it's iterned, which is an implementation detail, yes?

Not sure what you mean. That's a constant, so it'll always return the exact same object, surely?

I *think* that's only if it's interned -- and in any case, is a guarantee of the language, or an optimization?

I tried to test with a longer string, and it was the same one, but then I found in this arbitrary post on the internet:

... in Python 3.7, this has been changed to 4096 characters

( I guess I haven't played with that since 3.7) -- I haven't actually tried with a string literal linger than 4096 chars :-)

When you return a literal, like that, the literal becomes a function constant. That's not necessarily a language guarantee, but I would be hard-pressed to imagine a Python implementation that copied it for no reason.

But this certainly doesn't:

In [1]: def fun(): ...: return [1,2,3] ...:

In [2]: l1 = fun()

In [3]: l2 = fun()

In [4]: l1 is l2 Out[4]: False

So the issue is immutability and interning, not "literal display".

The difference is that list display isn't a literal, but a simple quoted string is. The square brackets are guaranteed to construct a new list every time.

My point is that a frozenset litteral could open the door to interning frozen sets, but that could probably be done anyway. And why? Are they heavily used in any code base?

A frozenset literal would allow the same frozenset to be reused, but it's not about interning. You're right that interning would be possible even without a literal, but in order to intern frozensets without a literal, we'd still need to construct a brand new (non-frozen) set to pass to the constructor, since there needs to be a way to indicate which elements we want. Technically, there are no "complex literals", but thanks to constant folding, a value like 3+4j can become a function constant. In theory, I could imagine a Python implementation that treats 3 and 4j as constants, but adds them at runtime, producing a unique complex object every time. But since there are no shadowable name lookups, it's easy enough to take advantage and gain a nice optimization. In theory, what could be done for frozensets would be to use a *tuple* for the arguments, which can then be used for an intern-style lookup: _frozenset = frozenset _fscache = {} def frozenset(t): if type(t) is tuple: # strict check, don't allow subclasses if t not in _fscache: _fscache[t] = _frozenset(t) return _fscache[t] return _frozenset(t) This would remove some of the copying, but at the cost of retaining all the tuples (in other words: now you have a classic caching problem). Could be useful for situations where it really truly is constant, but it's still a bit of a roundabout way to do things. A true frozenset literal can't possibly be shadowed, so the compiler would be free to do all the same optimizations that it does with tuples. ChrisA

On Mon, Jan 17, 2022 at 03:05:36AM +0000, MRAB wrote:

How about doubling-up the braces:

{{1, 2, 3}}

I mentioned that earlier. Its not *awful*, but as you point out yourself, it does run into the problem that nested sets suffer from brace overflow. # A set of sets. {{{{}}, {{1, 2}}, {{'a', {{}}, None}}}}

and for frozen dicts:

We don't even have a frozen dict in the stdlib, so I'm not going to discuss that here. If and when we get a frozen dict, if it is important enough to be a builtin, then we can debate syntax for it.

but how often would you do that?

Often enough: https://www.delftstack.com/howto/python/python-set-of-sets/ https://stackoverflow.com/questions/37105696/how-to-have-a-set-of-sets-in-py... https://stackoverflow.com/questions/5931291/how-can-i-create-a-set-of-sets-i... -- Steve

On Sun, Jan 16, 2022 at 10:33:41PM -0800, Christopher Barker wrote:

On Sun, Jan 16, 2022 at 7:05 PM Steven D'Aprano wrote:

...

I never suggested adding this "for consistency".

Then what ARE you suggesting it for?

Apologies if my initial post was not clear enough: https://mail.python.org/archives/list/python-ideas@python.org/message/GRMNMW... CPython already has all the machinery needed to create constant frozensets of constants at compile time. It already implicitly optimizes some expressions involving tuples and sets into frozensets, but such optimizations are fragile and refactoring your code can remove them. Ironically, that same optimization makes the explicit creation of a frozenset needlessly inefficient. See also b.p.o. #46393. The only thing we need in order to be able to explicitly create frozensets efficiently, without relying on fragile, implicit, implementation-dependent peephole optimizations which may or may not trigger, and without triggering the usual global+builtins name lookup, is (I think) syntax for a frozenset display. That would make the creation of frozensets more efficient, possibly encourage people who currently are writing slow and inefficient code like targets = (3, 5, 7, 11, 12, 18, 27, 28, 30, 35, 57, 88) if n in targets: do_something() to use a frozenset, as they probably should already be doing.

As far as I can tell, it would be a handy shorthand.

If you consider tuple, list and dict displays to be a handy shortcut, then I guess this would be too :-)

And you had suggested it could result in more efficient bytecode, but I think someone else thought that wasn't the case.

I see no reason why it wouldn't lead to more efficient bytecode, at least sometimes.

But what does that matter? are they heavenly used in any common code?

Personally, I think my code using frozensets is extremely heavenly :-) I doubt that frozensets are, or ever will be, as common as lists or dicts. In that sense, sets (frozen or otherwise) are, I guess, "Second Tier" data structures: - first tier are lists, tuples, dicts; - second tier are sets, deques etc. Or possibly "tier and a half" in that unlike deques they are builtins, which suggest that they are somewhat more important. In the top level of the stdlib (not dropping down into packages or subdirectories), I count 29 calls to frozenset. (Compared to 14 calls to deque, so on mere call count, I would say frozenset is twice as important as deque :-) Out of those 29 calls, I think that probably 13 would be good candidates to use a frozenset display form (almost half). For example: ast.py: binop_rassoc = frozenset(("**",)) # f{("**",)} asyncore.py: ignore_log_types = frozenset({'warning'}) # f{'warning'} Not all of them are purely literals, e.g. asyncore.py: _DISCONNECTED = frozenset({ECONNRESET, ENOTCONN, ...}) would still have to generate the frozenset at runtime, but it wouldn't need to look up the frozenset name to do so so there would still be some benefit. If we were particularly keen, that might go up to 19 out of the 29. The benefit is not huge. This is not list comprehensions or decorator syntax, which revolutionized the way we write Python, it is an incremental improvement. If the compiler didn't already have the machinery in place for building compile-time constant frozensets, this might not be worth the effort. But since we do, the cost of adding a frozenset display is relatively low (most of the work is already done, yes?) and so the benefit needs only to be small to justify the small(?) amount of work needed. -- Steve

On Sun, Jan 16, 2022 at 06:10:58PM -0800, Christopher Barker wrote:

In a way, what this might do is open the door to interning (some) frozen sets, like cPython does for some ints and strings.

I don't think that interning is relevant here. Interning is orthogonal to the existence of a frozenset display. You don't need to use a literal or display syntax to take advantage of interning: # At least some versions of CPython >>> m = int('78') >>> n = int('78') >>> m is n True And the existence of a literal or display syntax does not imply interning, e.g. floats, tuples. If there was a use for interning frozensets, we could do so regardless of whether or not there is a display syntax.

...

(using f{...} as a frozenset literal) does something different - it returns the *same* object, compiled once at function definition time, every time it's called.

why/how would it do that? It *could* do that -- as above, with interning.

Inside a function, CPython can cache literals and immutable displays made from purely literals, e.g. the tuple (1, 2, 3) but not the tuple (1, 2, x)) in co_consts. But we can't do any of that if the only way to create a frozenset is to lookup the name "frozenset" and call that function. So while not all frozensets inside a function could be built at compile-time and retrieved from co_consts, some of them could -- if only we had a display syntax equivalent to tuple displays. Ironically, if the compiler can prove that a regular mutable set display is only ever used once inside a function, CPython will make it a frozenset:

...

...

def func(a): ... return a in {1, 2, 4} ... func.__code__.co_consts (None, frozenset({1, 2, 4}))

So ironically the only way to put a constant frozenset into co_consts is to write it as a mutable set and only use it once!

We don't make changes to Python syntax unless there is a compelling reason.

"Compelling" depends on the cost of making the change. The bigger the cost, the more compelling the reason. The smaller the change, then the reason need not be as huge. I think this is a small change with a moderate benefit, which puts the cost vs benefit ratio on the benefit side.

There are any number of python types with no "literal" (well, not any number, it's quite defined. but still) heck, we don't even have literals for Decimal. Why this one?

Nearly all of those types are not builtins. As far as I can see, frozenset is the only commonly used, immutable, builtin type that doesn't have a literal display syntax, and the consequence of that is that code using frozensets does much more work than needed. -- Steve

On Mon, 17 Jan 2022 at 10:12, Steven D'Aprano wrote:

That would make the creation of frozensets more efficient, possibly encourage people who currently are writing slow and inefficient code like

targets = (3, 5, 7, 11, 12, 18, 27, 28, 30, 35, 57, 88) if n in targets: do_something()

to use a frozenset, as they probably should already be doing.

More realistically, would they not use a set already, as in targets = {3, 5, 7, 11, 12, 18, 27, 28, 30, 35, 57, 88} if n in targets: do_something() ? Is using a frozenset a significant improvement for that case? Because I doubt that anyone currently using a tuple would suddenly switch to a frozenset, if they haven't already switched to a set. Sure, there might be the odd person who sees the release notes and is prompted by the mention of frozenset literals to re-think their code, but that's probably a vanishingly small proportion of the audience for this change. BTW, I should say that I'm actually +0.5 on the idea. It seems like a reasonable thing to want, and if an acceptable syntax can be found, then why not? But I doubt it's going to have a major impact either way. Paul

On Mon, Jan 17, 2022 at 7:10 PM Steven D'Aprano wrote:

Out of those 29 calls, I think that probably 13 would be good candidates to use a frozenset display form (almost half). For example:

ast.py: binop_rassoc = frozenset(("**",)) # f{("**",)} asyncore.py: ignore_log_types = frozenset({'warning'}) # f{'warning'}

Both are in class scope so the overhead is very small.

Not all of them are purely literals, e.g.

asyncore.py: _DISCONNECTED = frozenset({ECONNRESET, ENOTCONN, ...})

would still have to generate the frozenset at runtime, but it wouldn't need to look up the frozenset name to do so so there would still be some benefit.

Name lookup is faster than building set in most case. So I don't think cost to look name up is important at all. Proposed literal might have significant efficiency benefit only when: * It is used in the function scope. and, * It can not be optimized by the compiler now. I am not sure how many such usages in stdlib. Regards, -- Inada Naoki

On Mon, Jan 17, 2022 at 08:04:50PM +0900, Inada Naoki wrote:

Name lookup is faster than building set in most case. So I don't think cost to look name up is important at all.

But the cost to look up the name is *in addition* to building the set. If you saw this code in a review: t = tuple([1, 2, 3, 4, 5]) would you say "that is okay, because the name lookup is smaller than the cost of building the list"? I wouldn't. I would change the code to `(1, 2, 3, 4, 5)`.

Proposed literal might have significant efficiency benefit only when:

* It is used in the function scope. and, * It can not be optimized by the compiler now.

Sometimes, now, the compiler *pessimizes* the construction of the frozen set. See b.p.o #46393. -- Steve

On Mon, Jan 17, 2022 at 8:49 PM Steven D'Aprano wrote:

On Mon, Jan 17, 2022 at 08:04:50PM +0900, Inada Naoki wrote:

...

Name lookup is faster than building set in most case. So I don't think cost to look name up is important at all.

But the cost to look up the name is *in addition* to building the set.

I meant it is negligible so we can just ignore it while this discussion.

If you saw this code in a review:

t = tuple([1, 2, 3, 4, 5])

would you say "that is okay, because the name lookup is smaller than the cost of building the list"?

I wouldn't. I would change the code to `(1, 2, 3, 4, 5)`.

* I never said it. I just said just lookup cost is not good reason because you listed name lookup cost for rationale. Please stop strawman. * tuple construction is much faster than set construction. So name lookup speed is more important for tuple. * Constant tuple is much much frequently used than constant set.

...

Proposed literal might have significant efficiency benefit only when:

* It is used in the function scope. and, * It can not be optimized by the compiler now.

Sometimes, now, the compiler *pessimizes* the construction of the frozen set. See b.p.o #46393.

I saw. And I know all the discussions in the b.p.o. already. But how important it is for Python depends on how often it is used, especially in hot code. Regards, -- Inada Naoki

On Mon, Jan 17, 2022 at 11:18:13PM +0900, Inada Naoki wrote:

...

On Mon, Jan 17, 2022 at 8:49 PM Steven D'Aprano wrote:

...

On Mon, Jan 17, 2022 at 08:04:50PM +0900, Inada Naoki wrote:

...

Name lookup is faster than building set in most case. So I don't think cost to look name up is important at all.

But the cost to look up the name is *in addition* to building the set.

I meant it is negligible so we can just ignore it while this discussion.

On my computer, the name lookup is almost a quarter of the time to build a set:

[steve ~]$ python3.10 -m timeit "frozenset" 10000000 loops, best of 5: 24.4 nsec per loop [steve ~]$ python3.10 -m timeit "{1, 2, 3, 4, 5}" 2000000 loops, best of 5: 110 nsec per loop

and about 10% of the total time:

[steve ~]$ python3.10 -m timeit "frozenset({1, 2, 3, 4, 5})" 1000000 loops, best of 5: 237 nsec per loop

If I use a tuple instead of the set, it is about 12% of the total time:

[steve ~]$ python3.10 -m timeit "frozenset((1, 2, 3, 4, 5))" 2000000 loops, best of 5: 193 nsec per loop

So not negligible.

That is in the global scope, which will be much slower than a local scope. Global builtins do a hash table lookup; local lookups just follow a

El lun, 17 ene 2022 a las 7:11, Steven D'Aprano () escribió: pointer.

-- Steve _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YL2DW3... Code of Conduct: http://python.org/psf/codeofconduct/

On 16 Jan 2022, at 18:44, Christopher Barker wrote:

Is there no way to optimize the byte code without adding to the language?

Not that it’s a bad idea anyway, but I wonder if frozen sets are common enough to warrant a change.

Are there any performance advantages to a frozen set? I ask because I do often use sets that could be frozen, but don’t need to be. E.g. they don’t change, nor are they used as keys.

For example:

If flag in {‘the’, ‘allowable’, ‘flags’}: …

If a frozen set was even a little bit faster or used less memory, it would be nice to be able to create one directly.

Not really relevant for the discussion, but CPython automaticly creates a frozenset here (set display with immutable members) as an optimisation.

...

...

dis.dis("x in {1,2,3}") 1 0 LOAD_NAME 0 (x) 2 LOAD_CONST 0 (frozenset({1, 2, 3})) 4 CONTAINS_OP 0 6 RETURN_VALUE

AFAIK the primary advantage of doing this is that the frozenset gets created once instead of every time the expression is executed. Frozenset itself is not faster than a regular set. Ronald — Twitter / micro.blog: @ronaldoussoren Blog: https://blog.ronaldoussoren.net/

On Mon, Jan 17, 2022 at 9:07 AM Ronald Oussoren wrote:

For example:

If flag in {‘the’, ‘allowable’, ‘flags’}: …

If a frozen set was even a little bit faster or used less memory, it would be nice to be able to create one directly.

Not really relevant for the discussion, but CPython automaticly creates a frozenset here (set display with immutable members) as an optimisation.

I think it's quite relevant to the discussion, because as far as I can tell, better performance in particular cases is the primary motivator. Funny that this has come up -- not too long ago, I did some experiments with code like the above: and to the surprise of myself and some other long-time Pythonistas I work with, using sets, rather tha tuples in those kinds of constructs, e.g.: if something in <a small collection or literals>: was always as faster or faster with sets than tuples. That was surprising because we assumed that construction of a set would be slower than construction of a tuple. And that was probably the case ten years ago. The proof is in the pudding,so I never bothered to figure out why, but now I know :-) Back to the topic at hand -- IIUC, set constants are already optimized, so the only places having a frozenset display would be when it is a constant, and it has to be a frozenset, where a regular one won't do. And that would only be noticeable if it was in a function that didn't do much else, and was called often. And in that case, it could be put in the global scope to ameliorate some of that cost. I believe Stephens' point is that the benefit may be fairly small, but so is the cost. I'm not so sure. I kind of like the idea myself, and the cost does seem small, but I don't think we should underestimate the cost of even this small complexity increase in the language. Sure, folks don't have toeven know it exists to write fine code, but it would be one more thing that newbies will need to figure out when they see it in others' code. In fact, there' a lot of what I might call "Python Scripters" that aren't even familiar with the set display at all. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

On Mon, Jan 17, 2022 at 07:19:17AM -0800, Jelle Zijlstra wrote:

...

That is in the global scope, which will be much slower than a local scope.

Global builtins do a hash table lookup; local lookups just follow a pointer.

Yes, that is correct, frozenset is a builtin, not a local variable, so looking the name up does two hash table lookups: one in globals, and if the name hasn't been shadowed, a second in builtins. -- Steve

On Tue, Jan 18, 2022 at 10:02 AM Joao S. O. Bueno wrote:

...

but I don't think we should underestimate the cost of even this small complexity increase in the language.

Actually, I think _maybe_ in this case the "complexity increase" cost is _negative_. People might waste more time looking for a way of spelling a frozenset literal than just filling in "frozenset(....)". I for one, even knowing that the cost of writing "frozenset({1,2,3})" is negligible, would "feel" better there was a way to spell that without the needless conversions.

That said, an appropriate prefix for the {} just as we do for strigns would be nice, and I disagree that it would be a significant source for "bugs". The "@{" is a nice way out if people think "f{}" would be too close to "f()". And "<1,2,3>" just for frozensets are indeed overkill. We already do "literal prefixing" with `"` after all. and formally extending this prefix usage as needed for other literals seems like a nice path. But, as far as bikeshedding go, we also have "literal sufixing" (2.0j anyone?)- maybe "{1,2,3}f" ?

I have been following along with not much to comment but this response sparked something in me. After reading all the viewpoints I think I would be +1 on the basic idea, and a +1 on the postfix/suffix syntax just suggested... the other syntaxes I'm more of +0.5 I like the way the suffix FLOWS with the act of writing the program. When I write a set, I am primarily focused on *what I am going to put in it*, and whether or not it should be mutable is kind of a later thought/debate in my head after I have established what it contains. As a dumb example, if my task at hand is "I need to create a bag of sports balls", I am mostly thinking about what goes into that bag at first, so I will write that first:

...

...

{Ball("basketball"), Ball("soccer"), Ball("football"), Ball("golf")}

Now I get to the end of that line, and I then sort of naturally think "ok does it make sense to freeze this" after i know what is in it. With the postfix syntax, I then either type the f:

...

...

{Ball("basketball"), Ball("soccer"), Ball("football"), Ball("golf")}f

...or not. With a prefix type syntax, or a smooth bracket syntax, either: A. it takes slightly more "work' at this point to "convert" the set to a frozenset, OR B. i have to think about ahead of time-- before i have actually written what is in the set- whether it will be frozen, or not. In contrast, when you are deciding whether to write a list vs a tuple, you are deciding between two things that are fundamentally far more different IDEAS than a "bag of things, frozen or unfrozen". A list is very often more of an open ended stack than it is "an unfrozen tuple". A tuple is very often much more of an object that can be used as a dictionary key, or a member of a set, than it is a container of things (of course, it is a container of things, too). These differences make is a lot easier to choose, ahead of time, which one makes sense before you have even written the line of code. Maybe I'm making too much of this, but I really like the idea of deciding at the END of the set literal whether to tack on that "f". --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler

One thing to consider is if we're going to have a syntax capable of creating an empty frozenset, we need one that creates an empty set. if f{...} exists, then s{...} should also exist? Regards João Bernardo On Tue, Jan 18, 2022 at 2:59 PM Rob Cliffe via Python-ideas < python-ideas@python.org> wrote:

I'm +1 on the idea. I'm happy with the f{ ... } syntax (although I did suggest something else). We already have letter-prefixes, let's stick to them rather than adding something new (which conceivably might one day find another use). Best wishes Rob Cliffe

On 18/01/2022 15:53, Ricky Teachey wrote:

On Tue, Jan 18, 2022 at 10:02 AM Joao S. O. Bueno wrote:

...

...

but I don't think we should underestimate the cost of even this small complexity increase in the language.

Actually, I think _maybe_ in this case the "complexity increase" cost is _negative_. People might waste more time looking for a way of spelling a frozenset literal than just filling in "frozenset(....)". I for one, even knowing that the cost of writing "frozenset({1,2,3})" is negligible, would "feel" better there was a way to spell that without the needless conversions.

That said, an appropriate prefix for the {} just as we do for strigns would be nice, and I disagree that it would be a significant source for "bugs". The "@{" is a nice way out if people think "f{}" would be too close to "f()". And "<1,2,3>" just for frozensets are indeed overkill. We already do "literal prefixing" with `"` after all. and formally extending this prefix usage as needed for other literals seems like a nice path. But, as far as bikeshedding go, we also have "literal sufixing" (2.0j anyone?)- maybe "{1,2,3}f" ?

I have been following along with not much to comment but this response sparked something in me.

After reading all the viewpoints I think I would be +1 on the basic idea, and a +1 on the postfix/suffix syntax just suggested... the other syntaxes I'm more of +0.5

I like the way the suffix FLOWS with the act of writing the program. When I write a set, I am primarily focused on *what I am going to put in it*, and whether or not it should be mutable is kind of a later thought/debate in my head after I have established what it contains.

As a dumb example, if my task at hand is "I need to create a bag of sports balls", I am mostly thinking about what goes into that bag at first, so I will write that first:

...

...

...

{Ball("basketball"), Ball("soccer"), Ball("football"), Ball("golf")}

Now I get to the end of that line, and I then sort of naturally think "ok does it make sense to freeze this" after i know what is in it. With the postfix syntax, I then either type the f:

...

...

...

{Ball("basketball"), Ball("soccer"), Ball("football"), Ball("golf")}f

...or not. With a prefix type syntax, or a smooth bracket syntax, either:

A. it takes slightly more "work' at this point to "convert" the set to a frozenset, OR B. i have to think about ahead of time-- before i have actually written what is in the set- whether it will be frozen, or not.

In contrast, when you are deciding whether to write a list vs a tuple, you are deciding between two things that are fundamentally far more different IDEAS than a "bag of things, frozen or unfrozen". A list is very often more of an open ended stack than it is "an unfrozen tuple". A tuple is very often much more of an object that can be used as a dictionary key, or a member of a set, than it is a container of things (of course, it is a container of things, too). These differences make is a lot easier to choose, ahead of time, which one makes sense before you have even written the line of code.

Maybe I'm making too much of this, but I really like the idea of deciding at the END of the set literal whether to tack on that "f".

--- Ricky.

"I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.orghttps://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/T7X3XI... Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/VRGSQR... Code of Conduct: http://python.org/psf/codeofconduct/

On 2022-01-18 18:54, Neil Girdhar wrote:

Even if f{1} creates a frozenset, I don't think f{} should create a frozenset.  I think it makes more sense to keep f{1: 2} open for frozendict if it ever makes it in.  Also, {} should be consisten with f{} (both should create dicts).  If you want an empty frozenset, you would have to do it the same way you do it for sets: either frozenset() or f{*()}.

[snip] An alternative might be to allow {,} for an empty set, which would then let you have f{,} for an empty frozenset.

On Tuesday, January 18, 2022 at 1:19:30 PM UTC-5 João Bernardo wrote:

One thing to consider is if we're going to have a syntax capable of creating an empty frozenset, we need one that creates an empty set.

if f{...} exists, then s{...} should also exist?

Regards João Bernardo

On Tue, Jan 18, 2022 at 2:59 PM Rob Cliffe via Python-ideas wrote:

I'm +1 on the idea. I'm happy with the     f{ ... } syntax (although I did suggest something else). We already have letter-prefixes, let's stick to them rather than adding something new (which conceivably might one day find another use). Best wishes Rob Cliffe

On 18/01/2022 15:53, Ricky Teachey wrote:

...

On Tue, Jan 18, 2022 at 10:02 AM Joao S. O. Bueno wrote:

>  but I don't think we should underestimate the cost of even this small complexity increase in the language.

Actually, I think _maybe_ in this case the "complexity increase" cost is _negative_. People might waste more time looking for a way of spelling a frozenset literal than just filling in "frozenset(....)". I for one, even knowing that the cost of writing "frozenset({1,2,3})" is negligible, would "feel" better there was a way to spell that without the needless conversions.

That said, an appropriate prefix for the {} just as we do for strigns would be nice, and I disagree that it would be a significant source for "bugs". The "@{" is a nice way out if people think "f{}" would be too close to "f()". And "<1,2,3>" just for frozensets are indeed overkill. We already do "literal prefixing" with `"` after all. and formally extending this prefix usage as needed for other literals seems like a nice path. But, as far as bikeshedding go, we also have "literal sufixing" (2.0j anyone?)- maybe "{1,2,3}f" ?

I have been following along with not much to comment but this response sparked something in me.

After reading all the viewpoints I think I would be +1 on the basic idea, and a +1 on the postfix/suffix syntax just suggested... the other syntaxes I'm more of +0.5

I like the way the suffix FLOWS with the act of writing the program. When I write a set, I am primarily focused on /what I am going to put in it/, and whether or not it should be mutable is kind of a later thought/debate in my head after I have established what it contains.

As a dumb example, if my task at hand is "I need to create a bag of sports balls", I am mostly thinking about what goes into that bag at first, so I will write that first:

   >>> {Ball("basketball"), Ball("soccer"), Ball("football"), Ball("golf")}

Now I get to the end of that line, and I then sort of naturally think "ok does it make sense to freeze this" after i know what is in it.  With the postfix syntax, I then either type the f:

   >>> {Ball("basketball"), Ball("soccer"), Ball("football"), Ball("golf")}f

...or not. With a prefix type syntax, or a smooth bracket syntax, either:

A. it takes slightly more "work' at this point to "convert" the set to a frozenset, OR B. i have to think about ahead of time-- before i have actually written what is in the set- whether it will be frozen, or not.

In contrast, when you are deciding whether to write a list vs a tuple, you are deciding between two things that are fundamentally far more different IDEAS than a "bag of things, frozen or unfrozen". A list is very often more of an open ended stack than it is "an unfrozen tuple". A tuple is very often much more of an object that can be used as a dictionary key, or a member of a set, than it is a container of things (of course, it is a container of things, too). These differences make is a lot easier to choose, ahead of time, which one makes sense before you have even written the line of code.

Maybe I'm making too much of this, but I really like the idea of deciding at the END of the set literal whether to tack on that "f".

Not a huge fan of an f-prefix for a frozen set (I prefer just recognizing the case and optimizing the byte code, I don't think frozensets are used often enough to justify its own syntax), but I love {,} for an empty set. On Tue, Jan 18, 2022 at 4:13 PM Rob Cliffe via Python-ideas < python-ideas@python.org> wrote:

On 18/01/2022 19:42, MRAB wrote:

...

On 2022-01-18 18:54, Neil Girdhar wrote:

...

Even if f{1} creates a frozenset, I don't think f{} should create a frozenset. I think it makes more sense to keep f{1: 2} open for frozendict if it ever makes it in. Also, {} should be consisten with f{} (both should create dicts). If you want an empty frozenset, you would have to do it the same way you do it for sets: either frozenset() or f{*()}.

[snip] An alternative might be to allow {,} for an empty set, which would then let you have f{,} for an empty frozenset. I like it! Rob Cliffe

---

Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/HXJXYO... Code of Conduct: http://python.org/psf/codeofconduct/

-- -Dr. Jon Crall (him)

On 19/01/22 6:41 am, Rob Cliffe via Python-ideas wrote:

I'm happy with the     f{ ... }

Fine with me too. I'd also be happy with making frozenset a keyword. It's hard to imagine it breaking any existing code, it avoids having to make any syntax changes, and all current uses of frozenset() on a constant set would immediately benefit from it. -- Greg

On 2022-01-18 23:50, Steven D'Aprano wrote:

On Wed, Jan 19, 2022 at 11:30:36AM +1300, Greg Ewing wrote:

...

I'd also be happy with making frozenset a keyword.

- int, float, str, tuple, dict, set, exceptions, len, etc are not keywords, so they can be shadowed (for good or bad);

- alone out of all the builtin types and functions, frozenset is a keyword.

Shadowing of builtin functions is a double-edged feature. But I think that, having made the decision to make them *not* keywords, they should either *all* be keywords, or *none*. It is weird to have some of them shadowable and some of them not.

None, True and False are special values, in a category of their own, so I don't think the inconsistency there is important. But having frozenset _alone_ out of the builtin functions and types a keyword would be a real Wat? moment.

I know about "foolish consistency", but then there is also foolish inconsistency. If we could just make one builtin function/type a keyword, with all the optimization advantages that allows for, would we *really* choose frozenset as the most important?

I don't know, the answer isn't clear to me. But it certainly wouldn't be my first choice.

A suggestion (that you're probably not going to like!) would be to have a way of indicating explicitly that you're referring a builtin, e.g. `frozenset` (using backticks). You could redefine "frozenset", but `frozenset` still refers to the builtin "frozenset".

+1 On Wed, 2022-01-19 at 07:20 +0000, Ben Rudiak-Gould wrote:

My preferred syntax for a frozenset literal would be something like

    {1, 2, 3}.freeze()

This requires no new syntax, and can be safely optimized at compile time (as far as I can tell).

set.freeze would be a new method of sets which could also be used at run time. It would return a new frozenset object and wouldn't alter the set object (so perhaps the name I suggested isn't ideal). Of course frozenset.freeze would just return itself.

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LYGTXE... Code of Conduct: http://python.org/psf/codeofconduct/

On Wed, Jan 19, 2022 at 07:20:12AM +0000, Ben Rudiak-Gould wrote:

My preferred syntax for a frozenset literal would be something like

{1, 2, 3}.freeze()

This requires no new syntax, and can be safely optimized at compile time (as far as I can tell).

I like that, it is similar to a proposal for docstrings: https://bugs.python.org/issue36906 This would be safe to optimize at compile time, so long as the contents of the set were all literals. And for implementations that didn't optimize at compile time, it would be no worse than the situation now. Perhaps a better name would be "frozen()` since that doesn't imply an in-place operation like "freeze" does. -- Steve

On Wed, Jan 19, 2022 at 07:12:04AM -0500, Ricky Teachey wrote:

Why does it need to be called at all?

{1, 2, 3}.frozen

For the same reason that most methods are methods, not properties. The aim of a good API is not to minimize the amount of typing, it is to communicate the *meaning* of the code as best as possible. `{1, 2, 3}.frozen` says that the result is an attribute (property, member) of the set. Like *name* to a person, or *tail* to a dog, the attribute API represents something which is part of, or a quality of, the object. The frozenset is not an attribute of the set, it is a transformation of the set into a different type. A transformation should be written as an explicit function or method call, not as attribute access. Yes, we can hide that transformation behind a property just to save typing two characters, but that is an abuse of notation. Also, it is standard in Python to avoid properties if the computation could be expensive. Copying a large set or millions of elements into a frozenset could be expensive, so we should keep it a method call. -- Steve

On Wed, Jan 19, 2022 at 6:21 AM Joao S. O. Bueno wrote:

Here is another hint that this usage would not resolve the problem of

...

having a literal frozenset. Even in the core of this discussion, with folks participating and knowing what they are talking about, the first thing that comes to mind when seeing a method call is that the target set would be copied.

sure, and in the general case, a_set.frozen() would presumably return a copy. as does frozenset(a_set) -- which is all we have at the moment.

But upon seeing a method call, we can just think first of a runtime behavior Correctly, btw. Any optimization there would be an exception, that people would have to know by heart.

But people shouldn't have to think about to trigger a compiler optimization anyway, that's just a nice bonus the compiler does for you. If this does all come to pass, then: s = {3,8,2}.frozen() will be slightly faster, in some case, than s = frozenset({3,8,2} but the result would be the same. There are plenty of tricks to in python to get a touch more performance, this would just be one more and frankly pretty rare that it would make an noticable difference at all. +1 on this +0 on f{} -1 on making frozenset a keyword -CHB On Wed, Jan 19, 2022 at 10:31 AM Steven D'Aprano

wrote:

...

On Wed, Jan 19, 2022 at 07:12:04AM -0500, Ricky Teachey wrote:

...

Why does it need to be called at all?

{1, 2, 3}.frozen

For the same reason that most methods are methods, not properties.

The aim of a good API is not to minimize the amount of typing, it is to communicate the *meaning* of the code as best as possible.

`{1, 2, 3}.frozen` says that the result is an attribute (property, member) of the set. Like *name* to a person, or *tail* to a dog, the attribute API represents something which is part of, or a quality of, the object. The frozenset is not an attribute of the set, it is a transformation of the set into a different type.

A transformation should be written as an explicit function or method call, not as attribute access. Yes, we can hide that transformation behind a property just to save typing two characters, but that is an abuse of notation.

Also, it is standard in Python to avoid properties if the computation could be expensive. Copying a large set or millions of elements into a frozenset could be expensive, so we should keep it a method call.

-- Steve _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/WS5IXN... Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/T55L2J... Code of Conduct: http://python.org/psf/codeofconduct/

-- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

On Thu, Jan 20, 2022 at 3:35 AM Stephen J. Turnbull < stephenjturnbull@gmail.com> wrote:

Christopher Barker writes:

...

If this does all come to pass, then:

s = {3,8,2}.frozen() will be slightly faster, in some case, than s = frozenset({3,8,2}

but the result would be the same.

There are plenty of tricks to in python to get a touch more performance, this would just be one more and frankly pretty rare that it would make an noticable difference at all.

+1 on this +0 on f{} -1 on making frozenset a keyword

Stated better than I could, expresses my feelings exactly. Sticking to integers (in floats I'd be less than -0 on f{}), I'll go with Chris's ratings, too.

Steve

Another agreement with Chris' ratings: +1 for .frozen() +0 on f{} -1 on keyword for frozenset But that still leaves the literal for the empty set as a problem. I'm still not sure what I think about {,} as an empty set. I tend to think it looks like "empty dictionary" and so could be confusing. Perhaps something like set.frozen() or set().frozen() could be optimized? --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler

See also the rejected PEP 351. -- Eric

On Jan 20, 2022, at 7:21 PM, Cameron Simpson wrote:

Well, I've just waded through this discussion.

This all feels to me like a special case of "wanting a constant for bytecode". What is we had a "freeze" operator, eg:

|foo|

which would produce a frozen version of foo. I'm liking the vertical bars myself, I think because it feels vaguely slightly analogous to "absolute value". So (assuming we can squeeze it into the expression syntax):

|{1,2,3}|

always makes a direct frozen set on the same basis that:

x in {1,2,3}

directly makes a frozenset by expression inspection. Then

Paired with a __freeze__ dunder method, this applies to any type, not just sets. (Where appropriate of course.)

So:

|{1,2,3}| frozen set |[1,2,3]| tuple! |any-iterable| tuple! |{1:2, 3:4}| frozen dict

Ths saves us (a) inventing a special syntax for frozen sets and (b) gateways to freezing many things, starting with the obvious above, via the __freeze__ dunder method.

This feels more general and less bikeshedable.

My main question is: is the syntax unambiguous?

Cheers, Cameron Simpson _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2RW36I... Code of Conduct: http://python.org/psf/codeofconduct/

On 20/01/22 3:17 am, Joao S. O. Bueno wrote:

But upon seeing a method call, we can just think first of a runtime behavior Correctly, btw. Any optimization there would be an exception, that people would have to know by heart.

Frozensets are immutable, so nobody should be making any assumptions about whether equal frozensets are the same object or not -- just as with ints, strings, etc. It would be just as legitimate e.g. for "ABC".upper() to return the same string object. There is no exception here to be learned. -- Greg

On 20Jan2022 19:31, Eric V. Smith wrote:

See also the rejected PEP 351.

Ah. So close to my idea as to be indistinguishable. That's a shame. Thanks, Cameron Simpson

On Fri, Jan 21, 2022 at 11:18:27AM +1100, Cameron Simpson wrote:

Paired with a __freeze__ dunder method, this applies to any type, not just sets. (Where appropriate of course.)

So:

|{1,2,3}| frozen set |[1,2,3]| tuple! |any-iterable| tuple! |{1:2, 3:4}| frozen dict

A frozen "any iterable" is not necessarily a tuple. For example, a frozen binary tree should probably keep the tree structure and methods; a frozen dict.keys() should be a frozen set; and its not clear what a frozen iterator should do. Should it run the iterator to exhaustion? Seems odd. What about non-collections? What's a frozen re.MatchObject? -- Steve

On Thu, 20 Jan 2022 at 10:19, Ricky Teachey wrote:

On Thu, Jan 20, 2022 at 3:35 AM Stephen J. Turnbull < stephenjturnbull@gmail.com> wrote:

...

Christopher Barker writes:

...

If this does all come to pass, then:

s = {3,8,2}.frozen() will be slightly faster, in some case, than s = frozenset({3,8,2}

but the result would be the same.

There are plenty of tricks to in python to get a touch more performance, this would just be one more and frankly pretty rare that it would make an noticable difference at all.

+1 on this +0 on f{} -1 on making frozenset a keyword

Stated better than I could, expresses my feelings exactly. Sticking to integers (in floats I'd be less than -0 on f{}), I'll go with Chris's ratings, too.

Steve

Another agreement with Chris' ratings:

+1 for .frozen() +0 on f{} -1 on keyword for frozenset

I really don't understand (having read everything above) why anyone prefers {1,2,3}.frozen() over f{1,2,3}. Yes, some people coming from some other languages might get confused (e.g. in Mathematica this is function call syntax) but that's true of anything: you have to learn Python syntax to use Python. The fact that {1,2,3} is a set and f{1,2,3} is a frozenset is not difficult to explain or to understand, especially in a language that already uses single letter prefixes for other things. The .frozen() method is a strangely indirect way to achieve a minor optimisation. Outside of attempting to achieve that optimisation it's basically useless because any time you would have written obj.frozen() you could have simply written frozenset(obj) so it does nothing to improve code that uses frozensets. With f{...} you have a nice syntax that clearly creates a frozenset directly and that can be used for repr. This is some actual code that I recently wrote using frozensets to represent monomials in a sparse representation of a multivariate polynomial:

...

...

poly = {frozenset([(1,2), (3,4)]): 2, frozenset([(0,1)]): 3} poly {frozenset({(1, 2), (3, 4)}): 2, frozenset({(0, 1)}): 3}

With the f{...} proposal you have actual syntax for this:

...

...

poly = {f{(1,2), (3,4)}: 2, f{(0,1)}): 3} poly {f{(1, 2), (3, 4)}: 2, f{(0, 1)}): 3}

With .frozen() it's

...

...

poly = {{(1,2), (3,4)}.frozen(): 2, f{(0,1)}.frozen()): 3} poly ??? does the repr change?

That difference in code/repr may or may not seem like an improvement to different people but that should be the real point of discussion if talking about a frozenset literal. The performance impact of frozenset literals is not going to be noticeable in any real application. My polynomial class makes extensive use of frozensets and is something that I do need to be as fast as possible. I just looked through the code I have for that class and none of the performance sensitive routines could benefit from this because they all actually need to build their elements in a dict before converting to a frozenset anyway e.g.: def mul(self, other): """multiply two (frozenset) monomials""" powermap = dict(self) for g, n in other: other_n = powermap.get(g) if other_n is None: powermap[g] = n else: powermap_n = other_n + n if powermap_n: powermap[g] = powermap_n else: powermap.pop(g) return frozenset(powermap.items()) I've just profiled this and the call to frozenset is always dwarfed by the time taken in the preceding loop which shows how cheap converting between builtins is compared to pretty much any other code. If you're using literals then of necessity you are talking about small sets. Even just using a small set over a small tuple is a hardly noticeable difference in speed in most situations: In [12]: s = {1,2,3} In [13]: t = (1,2,3) In [14]: timeit 2 in s 44.9 ns ± 0.17 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) In [15]: timeit 2 in t 59.9 ns ± 5.67 ns per loop (mean ± std. dev. of 7 runs, 10000000 loops each) -- Oscar

On Fri, Jan 21, 2022 at 6:57 AM Chris Angelico wrote:

On Fri, 21 Jan 2022 at 22:52, Ricky Teachey wrote:

...

On Fri, Jan 21, 2022 at 5:04 AM Steven D'Aprano

wrote:

...

...

On Fri, Jan 21, 2022 at 11:18:27AM +1100, Cameron Simpson wrote:

...

Paired with a __freeze__ dunder method, this applies to any type, not just sets. (Where appropriate of course.)

So:

|{1,2,3}| frozen set |[1,2,3]| tuple! |any-iterable| tuple! |{1:2, 3:4}| frozen dict

A frozen "any iterable" is not necessarily a tuple.

For example, a frozen binary tree should probably keep the tree structure and methods; a frozen dict.keys() should be a frozen set; and its not clear what a frozen iterator should do. Should it run the iterator to exhaustion? Seems odd.

What about non-collections? What's a frozen re.MatchObject?

-- Steve

lord have mercy, what a can of worms this could end up being!:

frozen iostream frozen property frozen function object frozen module object frozen iterator frozen datetime

Let's be fair here... The idea of freezing is to make it hashable, so there's no point talking about freezing a function, module, or datetime, since they are already hashable. Don't saddle the proposal with issues it doesn't have :)

(Technically this applies to an re.Match object too, actually, although I had to check to be sure. I've never once wanted to use one as a dict key. In contrast, I most certainly *have* used functions as dict keys, and it's safe and dependable.)

ChrisA

Great point! I learned something. --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler

On Fri, 21 Jan 2022 at 12:15, Chris Angelico wrote:

On Fri, 21 Jan 2022 at 22:52, Oscar Benjamin wrote:

...

I really don't understand (having read everything above) why anyone prefers {1,2,3}.frozen() over f{1,2,3}. Yes, some people coming from some other languages might get confused (e.g. in Mathematica this is function call syntax) but that's true of anything: you have to learn Python syntax to use Python. The fact that {1,2,3} is a set and f{1,2,3} is a frozenset is not difficult to explain or to understand, especially in a language that already uses single letter prefixes for other things.

The .frozen() method is a strangely indirect way to achieve a minor optimisation. Outside of attempting to achieve that optimisation it's basically useless because any time you would have written obj.frozen() you could have simply written frozenset(obj) so it does nothing to improve code that uses frozensets.

If set.frozen() is optimized, then str.upper() can be optimized the same way, which means there's a lot of places where constant folding can be used. We commonly write code like "7*24*60*60" to mean the number of seconds in a week, confident that it'll be exactly as fast as writing "604800", and there's no particular reason that method calls can't get the same optimization, other than that it hasn't been done yet.

While dedicated syntax might be as good, it also wouldn't help with string methods (or int methods - I don't see it a lot currently, but maybe (1234).to_bytes() could become more popular), and it would also be completely backward incompatible - you can't feature-test for syntax without a lot of hassle with imports and alternates. In contrast, code that wants to use set.frozen() can at least test for that with a simple try/except in the same module.

The proposal for .frozen() is not about optimising method calls on literals in general: the proposal is to add a method that is basically redundant but purely so that calls to the method can be optimised away.

Not one of the proposed syntaxes has seen any sort of strong support. This isn't the first time people have proposed a syntactic form for frozensets, and it never achieves sufficient consensus to move forward.

...

With f{...} you have a nice syntax that clearly creates a frozenset directly and that can be used for repr. This is some actual code that I recently wrote using frozensets to represent monomials in a sparse representation of a multivariate polynomial:

"Clearly" is subjective. Any syntax could be used for repr, including {1,2,3}.frozen(), so f{1,2,3} doesn't have any particular edge there. Personally, I think that string literals are not the same thing as tuple/list/dict/set displays, and letter prefixes are not as useful on the latter.

...

...

...

...

poly = {frozenset([(1,2), (3,4)]): 2, frozenset([(0,1)]): 3} poly {frozenset({(1, 2), (3, 4)}): 2, frozenset({(0, 1)}): 3}

With the f{...} proposal you have actual syntax for this:

...

...

...

poly = {f{(1,2), (3,4)}: 2, f{(0,1)}): 3} poly {f{(1, 2), (3, 4)}: 2, f{(0, 1)}): 3}

With .frozen() it's

...

...

...

poly = {{(1,2), (3,4)}.frozen(): 2, f{(0,1)}.frozen()): 3} poly ??? does the repr change?

Yes, it most certainly would change the repr. I don't see why that's an issue.

I'm not saying it's an issue. That was a genuine question. So I guess you'd expect this:

...

...

poly = {{(1,2), (3,4)}.frozen(): 2, f{(0,1)}.frozen()): 3} poly {{(1, 2), (3, 4)}.frozen(): 2, f{(0, 1)}.frozen()): 3}

This btw is my real point of my post which you seem to have missed (I probably should have kept it more direct):

...

That difference in code/repr may or may not seem like an improvement to different people but that should be the real point of discussion if talking about a frozenset literal. The performance impact of frozenset literals is not going to be noticeable in any real application.

If we take performance out of the equation would anyone actually propose to add a .frozen() method so that obj.frozen() could be used instead of frozenset(obj)? If so then what is the argument for having a redundant way of doing this?

I don't understand polynomials as frozensets. What's the point of representing them that way? Particularly if you're converting to and from dicts all the time, why not represent them as dicts? Or as some custom mapping type, if you need it to be hashable?

Hashability is the point. The polynomial is a dict mapping monomials to coefficients and the monomials are frozensets of factors so that they are hashable with unordered equality. Another option would just be a sorted tuple of tuples and then instead of frozenset(d.items()) you'd have tuple(sorted(d.items())) but that's slower in my timings (for all input sizes). Any custom type with pure Python __eq__/__hash__ methods would slow everything down because apart from a couple of functions like the one I showed all these objects are used for is as dict keys. -- Oscar

Yes, but have you _seen_ the bickering about the existing bracket

choices just for frozenset? Eww. Hence the going for a distinct operator altogether. Yes, I'd prefer brackets of some kind too, but they're taken.

If one uses prefixes, you start from 53 valid (all latin areas, upper,

lower and @) new brackets for {} . I can't see how they are "all taken" when the strongest argument against prefixing seems to be "but _only strings_ should have prefixes". (with the "typing f{} instead of f() is going to be a bug magnet" as a runner up). None of those stand up to any logical analysis It is ok voting that "the language should not be made more complex at this point, and we won't add any new syntax for a frozenset", but I think that if it is agreed that frozensets are ok, a prefix is just straightforward. And then, adopting prefixes for curly braces, you have 52 other bracket types to try and sell this "generic freezer operator" you are presenting here. :-). On Fri, Jan 21, 2022 at 5:52 AM Cameron Simpson wrote:

On 21Jan2022 01:16, MRAB wrote:

...

On 2022-01-21 00:18, Cameron Simpson wrote:

...

This all feels to me like a special case of "wanting a constant for bytecode". What is we had a "freeze" operator, eg: |foo| [...] Paired with a __freeze__ dunder method, this applies to any type, not just sets. (Where appropriate of course.) |{1,2,3}| frozen set |[1,2,3]| tuple! |any-iterable| tuple! |{1:2, 3:4}| frozen dict [...] My main question is: is the syntax unambiguous?

I don't know whether it's unambiguous, but it could be confusing.

For example, what does this mean: | a | b | ?

Yeah.

...

It's: | (a | b) | I think.

Probably. Running precedence the other way (or even worse, letting the valid combinations just shake out) would be confusing.

...

The problem is that '|' could be an opening '|', a closing '|', or an infix '|'.

You don't get this problem with differing open and closing pairs such as '(' and ')'.

Yes, but have you _seen_ the bickering about the existing bracket choices just for frozenset? Eww. Hence the going for a distinct operator altogether. Yes, I'd prefer brackets of some kind too, but they're taken.

Cheers, Cameron Simpson _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/FZN74P... Code of Conduct: http://python.org/psf/codeofconduct/

On Sat, 22 Jan 2022 at 00:30, Joao S. O. Bueno wrote:

...

Yes, but have you _seen_ the bickering about the existing bracket

choices just for frozenset? Eww. Hence the going for a distinct operator altogether. Yes, I'd prefer brackets of some kind too, but they're taken.

If one uses prefixes, you start from 53 valid (all latin areas, upper, lower and @) new brackets for {} .

Distinguishing upper and lower would be even more inconsistent, since strings don't. Or rather, they sorta-kinda do, but always define that x and X mean the same thing. The at sign is an operator, and cannot be used as a prefix. So you have 26, which is approximately 25 more than you will actually want, or maybe 23 if there's an explicit prefix for "set" and "dict" in there somewhere.

I can't see how they are "all taken" when the strongest argument against prefixing seems to be "but _only strings_ should have prefixes". (with the "typing f{} instead of f() is going to be a bug magnet" as a runner up). None of those stand up to any logical analysis

Nice how you are the judge of whether it stands up to logical analysis. Nice way to pooh-pooh an argument without any actual reasoning. Oh wait, that's exactly what logical analysis would be... So actually it's your rebuttal that doesn't stand logical analysis :)

It is ok voting that "the language should not be made more complex at this point, and we won't add any new syntax for a frozenset", but I think that if it is agreed that frozensets are ok, a prefix is just straightforward.

And then, adopting prefixes for curly braces, you have 52 other bracket types to try and sell this "generic freezer operator" you are presenting here. :-).

Prefixes on braces are not a generic freezer operator. An operator has to be able to act on any object, not a specific syntactic construct. Unless you're suggesting that q{obj} would mean "obj, but frozen", which would be a quite inconsistent protocol, given that it's doing the same job as a function call, looks disturbingly similar to a function call, but is actually an operator. I don't know of anything else in Python that behaves that way, except *maybe* the way a comma can create a tuple, or can be used as part of a series of things. (And that's a pretty weak line of argument, except in specific cases where it's actually ambiguous - see assert (x,y) for instance.) ChrisA

On Fri, Jan 21, 2022 at 3:52 AM Oscar Benjamin wrote:

...

...

...

s = {3,8,2}.frozen() will be slightly faster, in some case, than s = frozenset({3,8,2}

but the result would be the same.

I really don't understand (having read everything above) why anyone

If this does all come to pass, then: prefers {1,2,3}.frozen() over f{1,2,3}.

Because it doesn't require any change to the language -- it's a feature, not a language change. Every change to the language is s substantial burden to the whole community. Even a "small" one like this. It was absolutely worth for, e.g. f-strings, because they are a feature that has very broad use. This would have narrow use, and, I think even the OP said is more about potential optimization than a nicer syntax. If others think this syntax is substantially nicer, sure -- but then I'd argue that frozensets are simply not that commonly used -- you yourself realized that while you have an important use case, you aren't using literals that often anyway (at all?). Python is not a high-performance language -- has it ever had a feature added primirly so it could be optimized? (that is a serious question). And this does seem like a very small change, but is it? 1) folks, a couple years from now, reading new code might have never heard of a frozenset, and see: frozenset(something) or {1,5,2}.frozen() will probably have a pretty good idea what those mean, and if they have no clue, then it's easy to look up. f{3,1,6} not so much. And once we have ONE prefix on a bracket, I"ll bet you folks will suggest more ... small change medium churn tiny benefit (frankly, it's confusing enough that {a,b,c} makes a set and {} makes a dict, but what can we do? there's only so many brackets :-( -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

There seem to be two different reasons people want a generic freeze syntax: 1. Making a hashable copy of an arbitrary object 2. Avoiding O(n) rebuilding of literals on every use (a constant for bytecode, as you put it) In both 1 and 2, not only the object but all of its children need to be immutable. For 2, that's the status quo, but for 1 it seems like a bigger problem. There is already a solution of sorts for 1: pickle. It may even be more efficient than a subobject-by-subobject deep freeze since it stores the result contiguously in RAM. On the other hand it can't share storage with already-hashable objects. For the second one, I would rather have an "inline static" syntax (evaluates to the value of an anonymous global variable that is initialized on first use), since it would be more broadly useful, and the disadvantages seem minor. (The disadvantages I see are that it's built per run instead of per compile, it's theoretically mutable (but mutation would be evident at the sole use site), and it may use more heap space depending on how constants are implemented which I don't know.) As for the syntax... well, backticks, obviously...

On Fri, Jan 21, 2022 at 10:56:42PM +1100, Chris Angelico wrote:

Let's be fair here... The idea of freezing is to make it hashable,

And immutable.

so there's no point talking about freezing a function, module,

Neither of which are immutable. -- Steve

On Fri, Jan 21, 2022 at 08:36:37AM -0800, Christopher Barker wrote:

On Fri, Jan 21, 2022 at 3:52 AM Oscar Benjamin wrote:

...

I really don't understand (having read everything above) why anyone prefers {1,2,3}.frozen() over f{1,2,3}.

Because it doesn't require any change to the language -- it's a feature, not a language change. Every change to the language is s substantial burden to the whole community. Even a "small" one like this.

The change from "" ASCII byte strings to Unicode was a substaintial change that lead to a substantial burden on library developers, at least in the short-term. Adding a new operator like `@` has been barely noticed. Out of the two extremes, I don't think that adding f{} displays would be closer to changing the meaning of strings than to adding the matmul operator. I do acknowledge that f{} would be perplexing to beginners who just saw it written down with no explanation or context, but then the same applies to slicing and list comprehensions.

Python is not a high-performance language -- has it ever had a feature added primirly so it could be optimized? (that is a serious question).

Maybe we should stop dismissing performance as unimportant. Nobody has ever said that Python is too fast. I was going to joke that nobody has ever asked "How do I make Python slower?", but sure enough somebody has: https://stackoverflow.com/questions/16555120/how-can-i-slow-down-a-loop-in-p... *wink* I acknowledge that this specific change would likely be a micro- optimization, but looking beyond this proposal, perhaps we should start considering adding features specifically for performance? People are developing entire new Python interpreters and variants to improve performance, and have done so for many years. There is an impressive list of past and present such projects: - Cython - Nuitka - Pyston - Cinder - wpython - Hotpy - Unladen Swallow - Numba - Pythran among others. So maybe we should consider *not* saying "Python is fast enough", or "Just re-write it in C", and consider language features that are aimed specifically or mainly at performance. Just a thought to mull over. -- Steve

I hope the following remarks are constructive to the subject. It seems what's not liked about f{*x} notation is mainly that it looks parallel to f(*x) or e.g., often possibly f[(*x, )] , which seems to be a problem mostly only because of the character "f". If the prefix were "frozenset", then frozenset{*x} would be similar to frozenset(x) so it wouldn't be too wrong if the notation looks as if it's for a modification of a call of frozenset till this name is made to refer to something else by the programmer's own preference, even in which case, the name may not be very likely to refer to something conceptually far from the built-in frozenset. I also expect the notation wouldn't be too cryptic any more then. Best regards, Takuo Matsuoka

Hi Steve D'Aprano started this thread on 16 Jan, referencing https://bugs.python.org/issue46393. In the 95th message in this thread, on 27 Jan, Stephen J. Turnbull wrote: I think for many of us (specifically me, but I don't think I'm alone) it's

equally important that we aren't persuaded that there's a need for a frozenset literal great enough to overcome the normal reluctance to add syntax. A number of important cases are already optimized to frozensets, and use cases and (more important) benchmarks showing that this optimization is important enough to add syntax so the programmer can hand-code it are mostly to entirely lacking.

On 17 Jan, Serhiy Storchaka wrote to the b.p.o issue Steve D'Aprano referenced: As Steven have noted the compiler-time optimization is not applicable here

because name frozenset is resolved at run-time.

In these cases where a set of constants can be replaced with a frozenset of

constants (in "x in {1,2,3}" and in "for x in {1,2,3}") the compiler does it.

And I don't think there is an issue which is worth changing the language.

Creating a frozenset of constants is pretty rare, and it is even more rare in tight loops. The most common cases (which are pretty rare anyway) are already covered.

This is message 96 in this thread. Perhaps something bad (or good) will happen when we get to message 100. https://www.theregister.com/2022/01/27/linux_999_commits/ -- Jonathan

+1 I would appreciate being able to create a frozenset using f{1, 2, 3}. This will also make the `repr` of frozensets shorter. On Sun, Jan 16, 2022 at 10:33 AM Steven D'Aprano wrote:

Inspired by this enhancement request:

https://bugs.python.org/issue46393

I thought it might be time to revist the idea of a frozenset display. This has been discussed a few times before, such as here:

https://mail.python.org/pipermail/python-ideas/2018-July/051902.html

We have displays for the most important builtin data structures:

- lists [1, 2, 3] - tuples (1, 2, 3) - dicts {1: 0, 2: 0, 3: 0} - sets {1, 2, 3}

(as well as literals for ints, floats, strings and bytes)

but not for frozensets. So the only way to guarantee that you have a frozenset is to explicitly call the builtin, which is not only a runtime call rather than a compile-time operation, but can be monkey-patched or shadowed.

CPython already has some neat optimizations in place to use frozensets instead of sets, for example:

...

...

...

import dis dis.dis("x in {1, 2, 3}") 1 0 LOAD_NAME 0 (x) 2 LOAD_CONST 0 (frozenset({1, 2, 3})) 4 CONTAINS_OP 0 6 RETURN_VALUE

and the compiler can build frozensets of literals as a constant. Ironically, this means that actually making a frozenset explicitly does far more work than needed:

...

...

...

dis.dis("frozenset({1, 2, 3})") 1 0 LOAD_NAME 0 (frozenset) 2 BUILD_SET 0 4 LOAD_CONST 0 (frozenset({1, 2, 3})) 6 SET_UPDATE 1 8 CALL_FUNCTION 1 10 RETURN_VALUE

Got that? To create a frozenset of literals, first the compiler creates a frozenset constant containing what you wanted. Then at runtime, it:

- looks up frozenset in globals and builtins; - loads the pre-prepared frozenset (which is exactly what we want); - creates a new set from that frozenset; - calls the frozenset() function on that set to create a new frozenset that duplicates the pre-prepared one; - and finally garbage-collects the temporary set.

So to get the frozenset we want, we start with the frozenset we want, and make an unnecessary copy the long way o_O

If you didn't know that every step in that song and dance routine was necessary, it would seem ludicrously wasteful.

If we had a frozenset display, we could avoid most of that work, optimizing that down to a LOAD_CONST like this:

...

...

...

dis.dis('(1, 2, 3)') 1 0 LOAD_CONST 0 ((1, 2, 3)) 2 RETURN_VALUE

It seems to me that all of the machinery to make this work already exists. The compiler already knows how to create frozensets at compile-time, avoiding the need to lookup and call the frozenset() builtin. All we need is syntax for a frozenset display.

How does this work for you?

f{1, 2, 3}

-- Steve _______________________________________________ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-leave@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GRMNMW... Code of Conduct: http://python.org/psf/codeofconduct/

[... big snip...] On 22Jan2022 01:41, Chris Angelico wrote:

On Sat, 22 Jan 2022 at 00:56, Joao S. O. Bueno wrote:

...

At that point, I argue that despite adding still more things to the syntax, it is one that will spare time in average than the other way around, due to the time people, needing frozensets for the first time in any project, waste looking for a literal syntax for them only to find out there is not any.

Have you any stats on this? There is no literal/display syntax for datetimes, regular expressions, ranges, bytearrays, or a host of other common types. How often do people reach for a literal syntax for those? (I say "literal/display" since, technically, dicts have a display syntax, not a literal, and complex numbers are written as a constant-folded sum, but in practical terms, those count. The other types don't even get that.) If frozensets are so special that they need syntax, why not ranges, which are used far more frequently?

Well, some data. a) if the, for example, f{constant-set-display} syntax is generalisable (notionally, where we generalise it or not), it offers a path to frozen literals for other things via a prefix notation, should be become desirable. b) literal regexps: people use these _all the time_, conceptually. To the latter: Perl has literal regexps, you just write: /regexp-goes-here/ It even has a verbose commentable form of that to aid writing understandable regexps (hahaha!). What, we're not Perl? True, but you see _lots_ of code like this: # apologies if I have the argument order wrong here if re.match('regexp-string', 'target-string'): which effectively relies on the re module's autocaching of regexps to be efficient while skipping the more overt: # top of module foo_re = re.compile('regexp-string'[,options]) .... # in the main code if m := foo_re.match('target-string'): The former is nothing else but a metaphor for a literal regexp. I'm _not_ arguing for regexp literals in Python - IMO they're undesirable, a separate argument. (Note: not "undesired", just undesirable: to be avoided except when they're the right solution.) Cheers, Cameron Simpson

On Tue, 1 Feb 2022 at 09:02, Cameron Simpson wrote:

[... big snip...]

On 22Jan2022 01:41, Chris Angelico wrote:

...

On Sat, 22 Jan 2022 at 00:56, Joao S. O. Bueno wrote:

...

At that point, I argue that despite adding still more things to the syntax, it is one that will spare time in average than the other way around, due to the time people, needing frozensets for the first time in any project, waste looking for a literal syntax for them only to find out there is not any.

Have you any stats on this? There is no literal/display syntax for datetimes, regular expressions, ranges, bytearrays, or a host of other common types. How often do people reach for a literal syntax for those? (I say "literal/display" since, technically, dicts have a display syntax, not a literal, and complex numbers are written as a constant-folded sum, but in practical terms, those count. The other types don't even get that.) If frozensets are so special that they need syntax, why not ranges, which are used far more frequently?

Well, some data.

a) if the, for example, f{constant-set-display} syntax is generalisable (notionally, where we generalise it or not), it offers a path to frozen literals for other things via a prefix notation, should be become desirable.

b) literal regexps: people use these _all the time_, conceptually.

In other words: If we create literals for a bunch of different things, then frozensets would be neat, but if we don't, then other types should have priority. Is that correct? I'm of the opinion that range objects should get literal syntax before frozensets do. You're of the opinion that regexps should get literal syntax before frozensets do. We're broadly in agreement here. I would *much* rather see {1,2,3}.frozen() be constant-foldable than f{1,2,3} as a literal. (That said: I'm actually not convinced that regexps need literal syntax, because it wouldn't benefit alternate regexp engines on PyPI.) ChrisA