Re: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses
Are "asdict" APIs all that common?
There are at least two in the std lib: namedtuple._asdict(), and dataclasses.asdict. I assume there are others I'm not aware of.
Conventionally you would get the caller to do this... Anything else involves lots of guesswork...
I get the impression that what I am suggesting wasn't clear. Let me approach it from a different direction. The language already provides the mapping protocol for user defined dict-making; ie, dict(myobj). So "this is how you define dict-making" rabbit hole is not only not worth going down, it's already been addressed for a very long time. The problem I want to address is very specific and two pronged: 1. some of the std lib modules DON'T USE the mapping protocol (for various very legitimate reasons) 2. some user-defined types also don't use the mapping protocol, also often for legit reasons Therefore the proposal is not to guess anything at all. The idea is to rigorously track all the existing dict APIs that have been put into the std lib over the years into a dict-making convenience function, since, for example, dict(my_dataclass) and dict(my_namedtuple) are (legitimately) broken. One can then confidently apply that asdict() convenience function without having to resort to a complicated string of if statements that might not even catch all cases. The language already does this kind of cataloging for other things, like getting a list of all the reserved keywords. Why not catalog all the std lib dict-making APIs into a dict-making convenience function, too? For user-defined protocols, we simply leave HOW to dict-ify to the caller via the factory_dict argument (if it is provided). If it isn't provided, one of the following would happen: - raise an error -- similar or even identical to, perhaps, the error raised for dict("foo") or dict(42) - use the default dict_factory (which is dict; identical to above, but succeeds if there is a dict-casting protocol already implemented) I favor the second, but I'm not dogmatic about which of those would be better. Again: no interest in trying to guess at user defined dict-making APIs as such, aside from allowing the caller to provide that API themselves if they so wish.
On 04/05/2019 21:58, Ricky Teachey wrote:
dict(myobj) is only defined for mapping objects. For everything else, "dict-making" *isn't* addressed. (Also, it's not a protocol. I think what you want is a protocol, but you aren't specifying it well enough for me to grasp.)
Actually dict() would be the place to do this, surely? If all you want is to add dataclasses and namedtuples to the list of things dict() copes with, that's entirely reasonable. If you have a list of obviously mapping-like things dict() should cope with, that's fine. If you want someone else to go do the work for you, that's less fine, but hey, someone might think it's an interesting thing to do. If you want dict() to cope with arbitrary objects (which you still appear to), you need to say how.
I still think this is an unhelpful starting place. I would actually be hesitant to use an API that just said "oh, throw anything at me, I'll cope." I'd much rather use something that told me what format it expected data in, what I need to conform to, and do any conversion myself. With half an ounce of luck I'll be able to arrange to get the data in the right format in the first place and avoid a whole unnecessary layer of processing.
For user-defined protocols, we simply leave HOW to dict-ify to the caller via the factory_dict argument (if it is provided).
I am minded of the Larson cartoon, "...and now a miracle occurs..." This is the first time you've mentioned this argument (to dict(), presumably?).
Right. So what you want is to add a keyword argument to dict() such that dict(obj, factory_dict=dict_me_harder) calls dict_me_harder(obj) if obj is not a type dict() knows how to deal with natively, expecting it to return a new dict or raise an exception. Is that right? -- Rhodri James *-* Kynesim Ltd
Here I was talking about the "mapping protocol" (define keys() and getitem()) but as you say, this essentially turns your object INTO a mapping. That isn't the same as some dict-making API (well I guess, unless you include returning two-tuples from iter). That was bad phrasing on my part.
If all you want
is to add dataclasses and namedtuples to the list of things dict() copes
At this point I am really exploring the problem and potential solutions. Your responses have been helpful. I do not have a list of obviously mapping-like things that eschew the mapping protocol, other than NT and dclses. It would be helpful to know if more of these exist. I'd be happy to try and look for them, but I don't really know how I'd do that. You bring up a good point-- dict() likely makes more sense for this than expanding the dataclasses.asdict(). Looking at your menu of choices (which I think hit on most of the potential solutions), I would change my initial approach to the problem and suggest instead that two ideas should be considered: - add NamedTuple._asdict() and dataclasses.asdict() to the list of things dict() copes with - add an alt constructor that behaves just like the dict() constructor, but "dicts harder", as you put it But I see two problems with doing it using the dict() constructor: - typing.NamedTuple and @dataclass classes can also define their own keys() and getitem()-- the behavior of which might diverge from their respective asdict() methods. So dict() would need to decide whether nt._asdict() and dataclasses.asdict() take precedence over the mapping protocol (I would suggest that IF the mapping protocol exists on an object, it would take precedence). - I am sure there is lots of code out there currently depending on errors occuring when dict() is applied to a NT. So of the two options, I think an alt constructor is far superior. The alt constructor would still need to decide the first precedence problem though. I am minded of the Larson cartoon, "...and now a miracle occurs..."
This is the first time you've mentioned this argument (to dict(), presumably?).
I would actually be hesitant to use an API that just said "oh, throw anything at me, I'll cope." Well my initial idea (now reconsidered) was to change the behavior of dataclasses.asdict(), which does already have a dict_factory kwarg. I was referring to that asdict() argument. However for me, adding a kwarg to dict() is not on the table; dict() makes use of **kwargs and reserving one for this purpose is surely a bad idea. That last comment makes sense to me, but the alternative- adding handling of NT and dclses to dict()- seems bad because of backward compatibility problem I mentioned above. Therefore because of the constructor **kwargs problem, and because I agree miracles are bad things in programming :), and because you are right that it is better for the user to present the data in an expected format, adopting a dict_factory kwarg is probably not worth bothering with. I'm to happy to abandon that.
On 09/05/2019 14:42, Ricky Teachey wrote:
The only thing I can think of is going through the standard library documentation (or the code if you prefer) and seeing what leaps out as dict-like but not actually a mapping. That sounds like a lot of work to me :-( -- Rhodri James *-* Kynesim Ltd
Another difficulty I just thought of: - a NT class- and a dcls, for that matter- could also be an iterable two-tuples that would currently get dict-ified using iteration. So a precedence rule- does iter() win? Or the asdict functions?- would be needed for this, too. The answer to that seems a lot less obvious to me (compared to the mapping protocol), since iterating k-v pairs is relying on a bit of a side effect, but that side effect has probably existed since the dict was created.
Frankly, I don't think an "as dict" protocol is worth it, but if others did think so, we could: define a new dunder: __as_mapping__ (I'm using "mapping", rather than "dict", to allow duck typing) Then the built-in dict() constructor would: check for an __as_mapping__ dunder: -- if there, it would call it and expect to get a mapping back that it could turn into a dict. -- if not, it would go through its current process ( you could perhaps specify a subset of the mapping interface -- maybe all you'd need is .items(), for instance -- so it would actually return an iterator of key-value pairs. after all, dict() can already consume that.) Meanwhile, any other code that wants to could also use the __as_mapping__ dunder to be able to work with an object as though it were a mapping, even if that isn't its usual interface. As far as I can see, the only downside of this is that it would means a new standard dunder, which is a substantial downside, as (I think) it essentially adds overhead in all sorts of places, plus ads "One more protocol" to a language already ripe with protocols. At least that's the message I got when I suggested adding a dunder for a sorting key method. -CHB On Thu, May 9, 2019 at 10:13 AM Ricky Teachey <ricky@teachey.org> wrote:
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
I like the __as_mapping__ idea a lot since it leaves it up to the user to determine how dict() does dict-making. But I understand how the overhead from adding a single function call could be very significant to something that is used as often as dict(). What about pairing a __mapping__ method with a mapping() function instead? mapping() would not need to be a keyword; it might make sense as part of, perhaps, the types library. types.mapping() would be significantly slower than dict() only sometimes-- and resource critical code could still rely on dict(). But it would also be more forgiving. And possibly even more useful (for the reasons I've tried to do a good job of outlining).
On 10/05/2019 21:22, Ricky Teachey wrote:
I come back to the point of what the use cases for this are. So far you have presented a situation in which you have made your life difficult in order to make other people's lives easier (though I'd contend it doesn't actually do that). If this is a common enough occurrence, that would justify adding the (relatively small and self-contained) functionality to dict(). I personally doubt it, but I'm ready to be proved wrong.
It would still require other types to add and maintain __mapping__ dunders. While the bar for this is lower than adding functionality to dict(), I'm still not sure you've got over it. -- Rhodri James *-* Kynesim Ltd
It would still require other types to add and maintain __mapping__ dunders.
Sure — but that is inherent in this whole idea — nothing other than the type object itself could possibly know how to dictify itself. Though yeah, that’s the key question— how often is this useful??? -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
Sure — but that is inherent in this whole idea — nothing other than the type object itself could possibly know how to dictify itself.
This seems to have an obvious solution: object.__as_mapping__() could simply call dict(self). Anything that uses the mapping protocol could then be mappified. Though yeah, that’s the key question— how often is this useful???
I'll try to make a quick case it is. Dataclasses have existed for barely a year. I don't know how widely used they are yet, by I sure use the beschtick out of them. The API is very nice and I fully expect them to be wildly successful. But the dclses API encourages the use of the asdict function rather than the mapping protocol-- if they become successful and widely used, it will become more and more inconvenient to have to test for multiple mapping APIs. And this isn't the first alt mapping API. NT is already incredibly successful (to the degree it has been an intention of the core devs to use it more and more in core python libraries), and as such there is a ton of if isinstance(obj,NT) obj._asdict() else dict(obj) code out there. Another dictish type in the std lib is types.SimpleNamespace, which is typically dictified (or rather, re-dictified) via yet another API, dict(obj.__dict__). The more we pile these APIs upon APIs, the more ridiculous it gets to create a flexible API of your own that allows the user to choose what kind of structure representation THEY want to provide.
Python allows for breaking out from a loop through 'break' and 'continue' keywords. It would be nice if it was possible to break many loop levels using one command. I propose two constructions for that: break break break break break break ... continue break continue break break continue ... And so on. Example: for i in range(10): for j in range(10): if i == 2 and j == 3: break break for i in range(10): for j in range(10): if i == 2 and j == 3: break continue if i == 7: break break Breaking out from many loops at once is a common practice, currently implemented through boolean testing and exceptions. This proposal would make it cleaner.
Dear Haael With python you can write for i, j in something (): # loop body This would cover both your examples. For the first, use itertools.product. For the second, write a custom iterator. -- Jonathan On Sat, 11 May 2019, 18:22 haael, <haael@interia.pl> wrote:
On Sat, 11 May 2019 at 18:22, haael <haael@interia.pl> wrote:
Breaking out from many loops at once is a common practice
Do you have evidence for this? In my experience, it's very rare (although I concede it would be *occasionally* convenient). In most cases where I'd consider breaking out of multiple levels of nested loop, I've tended to find that it's probably more readable to refactor the code to use a generator, or something like that. Paul
I don't love the 'break break' syntax idea. But I find myself wanting to break out of nested loops quite often. Very rarely could this be transformed to an 'itertools.permutation' de-nesting, albeit perhaps more often than I actually do that. My usual kludge is a sentinel 'break_outer' variable that gets set back and forth between True and False working the loops. Usually work a name that descriptive of the actual domain of the code. On Sat, May 11, 2019, 1:40 PM Paul Moore <p.f.moore@gmail.com> wrote:
On 11 May 2019, at 23:20, David Mertz <mertz@gnosis.cx> wrote:
I don't love the 'break break' syntax idea. But I find myself wanting to break out of nested loops quite often. Very rarely could this be transformed to an 'itertools.permutation' de-nesting, albeit perhaps more often than I actually do that.
Do you have an example of such loops ?
judicious use of the for loop "else" may help here, too -- I know I often forget it's there. -CHB On Sat, May 11, 2019 at 2:26 PM Adrien Ricocotam <ricocotam@gmail.com> wrote:
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Sun, May 12, 2019 at 3:45 AM Christopher Barker <pythonchb@gmail.com> wrote:
judicious use of the for loop "else" may help here, too -- I know I often
forget it's there. yeah, I often do this when I want to break twice but this method only supports breaking twice at once but this doesn't support break once and break twice semantics simultaneously. ``` for x in range(n): for y in range(m): break else: continue break ``` On Sun, May 12, 2019 at 3:45 AM Christopher Barker <pythonchb@gmail.com> wrote:
-- Regards Srinivas Devaki Software Developer at Zomato, New Delhi Phone: +91 9491 383 249 Telegram: @eightnoteight
On Sat, May 11, 2019 at 1:22 PM haael <haael@interia.pl> wrote:
I'm surprised no one has yet mentioned splitting off the loops into a function and using "return" to break out of those loops. If you nest the function in the spot where the loops would be and use nonlocal statements, you can even modify variables in the surrounding code (counters, etc.) or leave values behind. If you're opposed to a function and performance is not critical [1], you can also raise a custom "exception" at the multi-break spot and handle it at the appropriate place in the outer code. This is useful if you need to break out of a function plus additional nesting layers. [1] I mention performance here because if I recall correctly, exception handling is quite expensive and slow in Python.
On 5/11/2019 1:20 PM, haael wrote:
Breaking out from many loops at once is a common practice, currently implemented through boolean testing and exceptions.
Python exceptions generalize both 'return' and 'break' as methods for escaping nested contexts and are intended for generalized flow control.
This proposal would make it cleaner.
I actually think that the general method is 'cleaner', be separating 'leave here' and 'arrive here'. This proposal would mean typing fewer words in the special case where there are nested loops within one function. The cost is a) more syntax to learn and b) tightly tying together the two loops more tightly than might be necessary, and thereby inhibiting re-factoring. In the examples above, where the loops are inherently tied together, the two loops can, as noted by others, be reduced to one. If they are not inherently tied together, one might want to reduce the inner loop to a comprehension or wrap it in a new function so it can also be used elsewhere. If the inner loop raises, wrapping or unwrapping it, as needed for reusability or efficiency, does not affect the logic. -- Terry Jan Reedy
Terry reminds me of a common case I encounter that cannot be transformed into a loop over itertools.product(). E.g. for main in stuff: if thing_about(main): for detail in more_stuff(stuff, main): if seen_enough(detail): # break to outer somehow else: for detail in different_stuff(): if seen_enough(detail): # break to outer somehow On Sat, May 11, 2019, 6:58 PM Terry Reedy <tjreedy@udel.edu> wrote:
On Sun, May 12, 2019 at 12:43 PM David Mertz <mertz@gnosis.cx> wrote:
For that kind of loop, there's a different problem, which is the duplication. So I would start by rewriting that as: for main in stuff: if thing_about(main): details = more_stuff(stuff, main) else: details = different_stuff() for detail in details: if seen_enough(detail): ... I'm not sure what you mean by "break to outer somehow", though, so I don't know what you're actually needing here. Concrete examples would help. ChrisA
I remembered, there is a "labeled break" in Perl. https://perldoc.perl.org/functions/last.html On Sun, May 12, 2019 at 10:50 AM Chris Angelico <rosuav@gmail.com> wrote:
Ok, sure. But what if different seen_enough() conditions are in the two inner loops? By "break to outer" ... I mean, don't get any more 'main' from 'stuff'. I meant to dig through some real code, but forgot because I was writing code for work. This example on my tablet is the general pattern I often need though. And obviously I've solved it one way or another hundreds of times. I think a "labelled break" would be nice though. It might have even been a past PEP. On Sat, May 11, 2019, 10:51 PM Chris Angelico <rosuav@gmail.com> wrote:
On Sun, May 12, 2019 at 1:16 PM David Mertz <mertz@gnosis.cx> wrote:
Ok, sure. But what if different seen_enough() conditions are in the two inner loops? By "break to outer" ... I mean, don't get any more 'main' from 'stuff'. I meant to dig through some real code, but forgot because I was writing code for work. This example on my tablet is the general pattern I often need though. And obviously I've solved it one way or another hundreds of times.
I think a "labelled break" would be nice though. It might have even been a past PEP.
This is why concrete examples are much easier to discuss, heh. In any case, it's often possible to either (a) redefine the inner loop as some sort of container operation, or (b) redefine the outer operation as a function, and use "return". So it would be helpful to have an example that CAN'T be rewritten in one of those ways, to use as a compelling example. ChrisA
I'll try to find a concrete one tomorrow. I did this recently, but I might have factored it away somehow. The real life code has lots of extraneous parts that need to be removed or simplified though. Both because they aren't FLOSS, and because the details would muddy things. "Trimmed from real" won't look much different than "invented from scratch". And yes, OF COURSE everything CAN be rewritten. But that's true in regards to *every* proposed addition. The sentinel break_outer variable is often the was to do it without really big restructuring (but with a handful of ugly bookkeeping lines instead). On Sat, May 11, 2019, 11:22 PM Chris Angelico <rosuav@gmail.com> wrote:
On Sun, May 12, 2019 at 1:30 PM David Mertz <mertz@gnosis.cx> wrote:
I'll try to find a concrete one tomorrow. I did this recently, but I might have factored it away somehow. The real life code has lots of extraneous parts that need to be removed or simplified though. Both because they aren't FLOSS, and because the details would muddy things. "Trimmed from real" won't look much different than "invented from scratch".
And yes, OF COURSE everything CAN be rewritten. But that's true in regards to *every* proposed addition. The sentinel break_outer variable is often the was to do it without really big restructuring (but with a handful of ugly bookkeeping lines instead).
Yes - that's exactly the sort of thing that makes a great example. Show the way you'd LIKE to write it, and then show how you have to write it using current tools. If the latter is really ugly, requires finicky bookkeeping, or is error-prone, it's a good argument in favour of the new feature. ChrisA
To be clear in this thread, I don't think I'm really ADVOCATING for a multi-level break. My comments are simply noting that I personally fairly often encounter the situation where they would be useful. At the same time, I worry about Python gaining sometimes-useful features that complicate the overall language and bring it closer to Perl's "everyone can write in their own style." So to answer Chris' question... well, i did find something recent, but it has a ton of other extraneous stuff. But I've simplified to something that isn't that much different from my example from my tablet yesterday, but is fairly realistic still. I have an existing function that looks roughly like this: def find_needle_in_haystacks(): found_needle = False for haystack in glob.glob('path/to/stuff/*'): fh = open(fname) header = fh.readline() if get_format(header) == 'foo': for line in fh: status = process_foo(line) if status = -1: found_needle = True break elif get_format(header) == 'bar': for line in fh: status = process_bar(line) if status = -1: found_needle = True break if found_needle: break If I had a "labelled break" feature, I might rewrite it something like the following (picking convenient syntax that I'm not per-se advocating as best, even if the concept is adopted): # Hypothetical future labelled break: def find_needle_in_haystacks(): for haystack in glob.glob('path/to/stuff/*') label HAYSTACKS: fh = open(fname) header = fh.readline() if get_format(header) == 'foo': for line in fh: if process_foo(line) == -1: break HAYSTACKS elif get_format(header) == 'bar': for line in fh: if process_bar(line) == -1: break HAYSTACKS In any case, it's often possible to either (a) redefine the inner loop
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Sun, 12 May 2019 at 18:26, David Mertz <mertz@gnosis.cx> wrote:
Nice. Suggestion: 1. replace "label HAYSTACKS" with "as HAYSTACKS", if possible, so no new keyword is introduced; 2. replace "break HAYSTACKS" with "break from HAYSTACKS"
-- Gustavo J. A. M. Carneiro Gambit Research "The universe is always one step beyond logic." -- Frank Herbert
On Sun, May 12, 2019, 3:33 PM Gustavo Carneiro <gjcarneiro@gmail.com> wrote:
I thought of 'as' initially, and it reads well as English. But it felt to me like the meaning was too different from the other meanings of 'as' in Python. I might be persuaded otherwise. Likewise, 'from' isn't really very similar to other existing uses.
On Sun, 12 May 2019 at 21:06, David Mertz <mertz@gnosis.cx> wrote:
I thought of 'as' initially, and it reads well as English. But it felt to me like the meaning was too different from the other meanings of 'as' in Python. I might be persuaded otherwise.
If you think in terms of "named loops" rather than "labels", "as" makes a lot more sense. Having said that, I don't think this change is worth it, so debates about naming are mostly just theoretical to me :-) Paul
On Sun, May 12, 2019, 5:36 PM Paul Moore <p.f.moore@gmail.com> wrote:
Every name created with 'as' currently is a regular Python object, with a memory address, and some methods, that can participate in comparison (at least equality), that has a type, can be passed as a function argument, etc. Actually, all that is true of EVERY name in Python other than keywords. It seems like none of this would be true of a loop name/label. Hence my first thought that a different way of introducing the word is more clear... Maybe we can just prefix lines with labels instead, and add a 'goto' command :-)
How about doing it pythonic way then, "everything is a hack on a symbol table". Let normal loops remain normal loops. Let's introduce a special construction:
for x in iterator as loop_control_object: loop_body(loop_control_object)
The iterator in the loop would be wrapped inside another iterator that would yield everything as the original iterator, but it would also be controllable through loop_control_object. Sample implementation:
The control object could have methods that allow sophisticated loop control, such as breaking, continuing, injecting exceptions, counting iterations, skipping etc. The keywords 'break loop_control_object' and 'continue loop_control_object' could be simply implemented as:
loop_control_object.do_break()
I thought of "we could return immediately" shortly after I posted it. Yes, that's true in the example I wrote. But it's easy enough to vary it so that something happens after the loop as well. I also noticed that for the simple version, it might be slightly shorter to put the conditional inside the loop rather than to decide between loops. There are lots of ways to work around the absence of a labelled break. Many times putting the inner loop in a comprehension can simplify things too, for example (that's roughly what my actual code was refactored to, mostly not just because of the break issue). On Sun, May 12, 2019, 3:38 PM Chris Angelico <rosuav@gmail.com> wrote:
The concrete example I was working on when I started to miss double break. This is an implementation of polynomial long division in Galois field. Almost unmodified. With outer break, I would't need to use the `running` variable. In fact, for mathematical clarity, I would like to put a test in the outer loop, like: `while dividend`. It would make clear what this algorithm is actually doing. Sure, it could be refactored, but it would lose the relative simplicity. For such operations it is sometimes important to remain close to the raw mathematical notation. def __divmod__(self, other): dividend = self.canonical() try: divisor = other.canonical() except AttributeError: return NotImplemented if not dividend: return Field.zero(self.size), Field.zero(self.size) try: d = next(iter(divisor)) # leading term of the divisor except TypeError: d = self.const(divisor) do = self.__monomial_order(d) result = Field.zero(self.size) running = True while running: # outer loop, retry the changing `dividend` for x in dividend: # yield all monomials from this polynomial if self.__monomial_order(x) < do: # encountered monomial of too small order, finish running = False break # exit outer loop try: c = self.__monomial_division(x, d) # may fail assert c # sanity check result += c dividend -= c * divisor dividend = dividend.canonical() # if `dividend` nonzero, try again # if `dividend` is zero, finish running = bool(dividend) break except ArithmeticError: # monomial division failed, try next one continue # inner loop else: # `dividend` exhausted, finish running = False pass if not hasattr(result, 'operator'): result = self.const(result) return result.canonical(), dividend.canonical()
If I may propose `break n` as a replacement for the original message's `break break ... break`, where n>0 is the number of contiguous loops to break out of and `break 1` is synonymous with `break`. Seems easier on my eyes/noggin than counting out the individual `break` statements. Eli On Sat, May 11, 2019, 10:21 AM haael <haael@interia.pl> wrote:
On Sun, May 12, 2019 at 01:36:28AM -0700, Elias Tarhini <eltrhn@gmail.com> wrote:
This is very much error-prone because on any refactoring (increasing or decreasing the number of loop levels) one must increase/decrease all numbers in internal loops. Labels are at least stable.
Eli
Oleg. -- Oleg Broytman https://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.
On Sun, May 12, 2019 at 11:16:21AM +0200, Oleg Broytman wrote:
Indeed. Go has labels for loops: Label: for { ... break Label } as does Rust: 'label: while condition: { ... break label } We can't use the same syntax in Python, but we might write it like this: @label while condition: block break label and similar with for loops: @outer for x in range(100): @inner for y in range(3): if x == y == 1: break # like ``break inner`` if x == y == 2: break outer print(x, y) # break inner jumps to here # break outer jumps to here -- Steven
What if labels are placed in comments beside the loops, like how type is, at least, being used by analyzers? for x in range(100): # label: outer for y in range(3): # label: inner On Sun, May 12, 2019 at 5:45 PM Steven D'Aprano <steve@pearwood.info> wrote:
On 2019-05-12 10:44, Steven D'Aprano wrote:
@ reminds me too much of decorators. How about this: <label> while condition: block break label <outer> for x in range(100): <inner> for y in range(3): if x == y == 1: break # like ``break inner`` if x == y == 2: break outer print(x, y) # break inner jumps to here # break outer jumps to here
On 11/05/2019 02:46, Christopher Barker wrote:
Though yeah, that’s the key question— how often is this useful???
I have wanted something similar to this in Jython. Java supplies a battery of containers (with different performance and concurrency characteristics) and we monkepatch onto them the methods of list, set or dict according to their interface as java.util.List, Set or Map. This works "slightly badly", because sometimes those Python types have methods with the same name as one in the underlying container, but a different meaning. E.g. list.pop() disgrees about which end. I'd much rather have some kind of explicit wrapping or cast into their Python type: I had imagined the wrapped object would identify and behave as exactly that Python type (or ABC), however, it would be a *proxy* to the original. Thus one obtains a "view" onto the original collection, or looked at another way, a way to choose the implementation behind a particular Python dict, set or list. Several alternative ways of expressing this come to mind, but .asdict() is one. It's not always clear in this discussion whether a proxy is envisaged (where a put() goes through to the original object, if allowed) or just a way of constructing and filling a new dict. The latter would not fit my imagined use. Jeff Jeff Allen
On Sat, May 11, 2019 at 4:06 AM Jeff
The ABCs are already defined — if you want to make a wrapper proxy object, you implement the ABC you want to emulate. Now that I’m thinking about it more, this really covers most use cases: if you want your object to be converted to a dict by passing it to dict(), then implement the iteration protocol, returning key-value pairs that you want. This would work fine for dataclasses, and SimpleNamespaces, if one wanted to implement that. The only problem here is if you want to use the standard iteration protocol for something different -- I can't image what that would be. Sure, on the general case, maybe, but for a class that has a "natural" use case for dictifying, how else would you want to iterate over it?? -CHB
On 2019-05-11 12:50, Christopher Barker wrote:
One example would be. . . dicts. Calling dict on a dict gives you a dict, but iterating over the dict just gives you the keys. Certain other mapping types (such as Pandas Series and DataFrames) also have this behavior where iterating over the mapping gives you only the keys. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Sat, May 11, 2019 at 1:38 PM Brendan Barnwell <brenbarn@brenbarn.net> wrote:
well, there is that ... :-) And that IS the mapping protocol. I guess i was thinking that you had an object that was NOT a Mapping already. If it IS a Mapping, then presumably it already can be passed into the dict constructor, yes? So if you want your class "dictable", then you can implement either the Mapping ABC, or the iteration protocol. I"m having an even harder time imagining why you would want a Mapping interface that isn't the same as what you'd want turned into a dict. So why again do we yet another way to do it? Anyway, I was playing with this, and write a prototype decorator that makes a datacalss "dictable" -- that it, it can be passed in to the dict constructor to make a dict. This is done by providing an iterator that returns (key, value) pairs. The code is in this git rep, if anyone wants to comment on it or improve, it, or... https://github.com/PythonCHB/PythonTopics/blob/master/Sources/code/dict_maki... (also here for your reading pleasure) I'm not sure how the dict constructor decides an input is a mapping or an iterable -- maybe I'll play with that now. -CHB #!/usr/bin/env python """ A prototype of a decorator that adds an iterator to a dataclass so it can be passed in to the dict() constructor to make a dict. If this is thought to be useful it could be added to the dataclass decorator itself, to give all decorators this functionality. """ from dataclasses import dataclass class DataClassIterator: """ Iterator for dataclasses This used the class' __dataclass_fields__ dict to iterate through the fields and their values """ def __init__(self, dataclass_instance): self.dataclass_instance = dataclass_instance self.keys_iter = iter(dataclass_instance.__dataclass_fields__.keys()) def __iter__(self): return self def __next__(self): key = next(self.keys_iter) return (key, getattr(self.dataclass_instance, key)) def _dunder_iter(self): """ function used as the __iter__ method in the dictable_dataclass decorator """ return DataClassIterator(self) def dictable_dataclass(the_dataclass): """ class decorator for making a dataclass iterable in a way that is compatible with the dict() constructor """ the_dataclass.__iter__ = _dunder_iter return the_dataclass # Example from the dataclass docs: @dictable_dataclass @dataclass class InventoryItem: '''Class for keeping track of an item in inventory.''' name: str unit_price: float quantity_on_hand: int = 0 def total_cost(self) -> float: return self.unit_price * self.quantity_on_hand if __name__ == "__main__": # try it out: inv_item = InventoryItem("sneakers", 50.0, 20) print("an InventoryItem:\n", inv_item) print() print("And the dict you can make from it:") print(dict(inv_item))
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
OK, so some more exploration: Other may already know this, but it seems that the dict constructor essentially checks if the passed in object is a Mapping ABC: if it is, it iterates on teh keys and used __getitem__ to get the values. if it is not, it iterates over key, value pairs. So if you want your custom class to present the Mapping interface, and be able to be passed in to dict(), then you shoudl subclass from collections.Mapping.ABC If you do not want your class to present the Mapping interface, then your __Iter__ should return an iterator that returns key.value pairs. Here is some code that explores that with a custom Mapping class. https://github.com/PythonCHB/PythonTopics/blob/master/Sources/code/dict_maki... (also below) The trick here is that with, e.g. the dataclasses decorator, I'm not sure how to make it derive from the Mapping ABC -- I'm sure it's doable, but I haven't dug into that yet. Personally, I'm a big fan of duck typing -- so rather than check for the Mapping ABC, I think it should check for .items() first, and use that if it's there, and then use the regular iterator otherwise. Then all you'd have to do to make your class "dictable" would be to provide an items() iterator. But again -- there are two ways to make your classes dictable -- why do we need a third? -CHB Sample code: """ an exploration of how the dict constructor knows whether it is working with a MApping of a general iterable. It looks like the code is something like this: if isinstance(input, collections.abc.Mapping): self.update = {key: input[key] key in input} else: self.update = {key: val for key, val in input} So if your custom class present themapping interface -- that is, iterating over it returns the keys, than you want to be a MApping ABC subclass.ABC But if your custom class is not a Mapping, then you want __iter__ to return an iterator over teh key, value pairs. """ from collections.abc import Mapping def test_iter(instance): yield ("this") yield ("that") class DictTest(Mapping): def __init__(self, this="this", that="that"): self.this = this self.that = that def __iter__(self): print("iter called") return test_iter(self) def __getitem__(self, key): return getattr(self, key) def __len__(self): print("__len__ called") if __name__ == "__main__": dt = DictTest() print(dict(dt)) dt = DictTest(this=45, that=654) print(dict(dt)) On Sat, May 11, 2019 at 1:51 PM Christopher Barker <pythonchb@gmail.com> wrote:
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
It’s a design goal of dataclasses to not be iterable. https://www.python.org/dev/peps/pep-0557/#why-not-just-use-namedtuple Eric
Using iter() for the sole purpose of being able to shoehorn your user class into being dict-cast-able is, IMO, an abuse of iter. If your class makes logical sense as an iterable of key/value pairs, then for petesake, just use the mapping protocol (implement getitem and keys). iter() is for constructing sequences, not mappings. And if the class DOESN'T make logical sense as an interable of key/value pairs, it still isn't time to abuse iter() for the convenience of making dicts. Just make an asdict() method. The above, I believe, is great API advice that I didn't come up with myself-- and it's so great that it has been followed by both NT and dclses. But the downside is, we now have the inconvenience of multiple dict-making APIs.
On Sun, May 12, 2019 at 10:14 AM Ricky Teachey <ricky@teachey.org> wrote:
Using iter() for the sole purpose of being able to shoehorn your user class into being dict-cast-able is, IMO, an abuse of iter.
If your class makes logical sense as an iterable of key/value pairs, then for petesake, just use the mapping protocol (implement getitem and keys). iter() is for constructing sequences, not mappings.
iter() is for collections of all sorts; a sequence is specifically an ordered and indexed collection, but you can iterate over many things, including sets - and mappings. There's nothing wrong with iterable mappings, and in fact, I would be surprised (and not in a good way) by any that wasn't. ChrisA
On Sat, May 11, 2019 at 3:26 PM Eric V. Smith <eric@trueblade.com> wrote:
It’s a design goal of dataclasses to not be iterable.
https://www.python.org/dev/peps/pep-0557/#why-not-just-use-namedtuple
you would know, but that reference talks about why they are not the same as NamedTuple. if dataclasses *were*iterable, they almost certainly wouldn't iterate over the values alone. And NamesTuple was designed to BE tuples -- that is a drop in replacement for tuples -- so they have the features they have partially for that reason. And my toy code actually adds another decorator to make dataclasses iterable, so it would be a completely optional feature. All that being said, I'm not actually advocating doing this with dataclasses -- I'm simply making the point that there are currretly two ways to make custom class able to be passed in to the dict constructor -- do we need another? -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
That section mentions why they’re not iterable. Search on “iterable”.
if dataclasses *were*iterable, they almost certainly wouldn't iterate over the values alone.
That wouldn’t make a difference. The given NT example would still be a problem.
And NamesTuple was designed to BE tuples -- that is a drop in replacement for tuples -- so they have the features they have partially for that reason.
And my toy code actually adds another decorator to make dataclasses iterable, so it would be a completely optional feature.
That approach of adding support for iterability makes sense to me. I’m contemplating adding a “dataclasses_tools” package for some other tools I have. But I’m not sure how this fits in to the asdict discussion. Eric
On Sat, May 11, 2019 at 8:23 PM Eric V. Smith <eric@trueblade.com> wrote:
Sure, but in the context of the difference with NamedTuple: """ Instances are always iterable, which can make it difficult to add fields. If a library defines: Time = namedtuple('Time', ['hour', 'minute']) def get_time(): return Time(12, 0) Then if a user uses this code as: hour, minute = get_time() then it would not be possible to add a second field to Time without breaking the user's code. """ which is a great argument for why you wouldn't want dataclasses to iterate on the *values* of the fields, which is what NamedTuple does. In fact, beyond iteration, NamedTuple, IS a tuple, so: - the number of "fields" matters, - the order of the "fields" matters, - it's the values that really matter -- the field names are incidental. None of this (intentionally) applies to dataclasses - which is what i took as the point of that section of the PEP. But I see dataclasses as being more analogous to a dict than a tuple: - The field names matter - The order of the fields does not matter (the current state of the dict implementation not withstanding) So *if* dataclasses were to be iterable, then I'd think they should either implement the Mapping protocol (which would make them, similar to NamedTuple, a drop-in replacement for a dict, or iterate over (field_name, value) pairs. which would not introduce the issues staed in teh PEP with the iterability of NamedTuple. Note: I am NOT advocating for making dataclasses a Mapping -- I believe in a clear separation between "data" and "code" -- dicts are suitable for data, and dataclasses are suitable for code. Given PYthon's dynamic nature, this isn't always a clear boundary, but I think it's a better not to purposely make it even less clear. Others may have a different take on this -- it could be kind of a cool way to make a dict with extra functionality, but I think that's ill-advised.
you know, I'm embarrassed to say that I read through a lot of this thread wondering what the heck an NT is ;-) -- but now I got it. but I don't think the NT example is relevant here -- it's an issue with NT, because NTs are designed to be, and used as drop in replacements of tuples, and a lot of code out there does use tuples for "unpacking" assignments. But I'm going to hazard a guess that it's pretty rare for folks to do: def get_time(): ... return {"hour", 3, "minute", 5} hour, minute = get_time() and: result = get_time() hour = result['hour'] minute = results["minute"] wouldn't break, now would: time_dict = dict(get_time()) Anyway, we need dataclasses AND NamedTuples because dataclasses down't behave like a tuple -- but that doesn't mean that they couldn't behave more like a dict in some ways.
I like that -- good idea. If any of my toy code is useful, please feel free to make use of it.
But I’m not sure how this fits in to the asdict discussion.
Well, it fits in in the sense that the question at hand is: Is there a need for a standard protocol for making dicts out of arbitrary classes? datacalsses were used as an example use-case, as datacalsses do naturally map to dicts. My point is that there are already two protocols for making an arbitrary classes into a dict, so I don't think we need another one. Even though those two are overloading other use cases: the Mapping ABC and iteration. This does mean that there would be no way to shoehorn NamedTuples into this -- as they already use the iteration protocol in a way that is incompatible with dict creation. So yeah, maybe a use case there. But the other option is to use a convention, rather than an official protocol -- if an object has an "asdict()" method, then that method returns a dict. pretty simple. If you were dealing with arbitrary objects, you'd have to write code like: try: a_dict = an_object.asdict() except (AttributeError, Type Error): a_dict = something_else but you'd need to do that anyway, as you couldn't count on every type being conformant with the dict() constructor anyway. So I'm still trying to see the use cases for having wanting to turn arbitrary objects into dicts. As for NamedTuple -- it does have _asdict: In [122]: p._asdict() Out[122]: OrderedDict([('x', 4), ('y', 5)]) (sidenote --now that dicts preserve order, maybe NamedTuple could return a regular dict) So I see the OP's point -- and it's a leading dunder name because: """ To prevent conflicts with field names, the method and attribute names start with an underscore. """ which I suppose means that if there is an informal protocol, it should also use that leading underscore. -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On 04/05/2019 21:58, Ricky Teachey wrote:
dict(myobj) is only defined for mapping objects. For everything else, "dict-making" *isn't* addressed. (Also, it's not a protocol. I think what you want is a protocol, but you aren't specifying it well enough for me to grasp.)
Actually dict() would be the place to do this, surely? If all you want is to add dataclasses and namedtuples to the list of things dict() copes with, that's entirely reasonable. If you have a list of obviously mapping-like things dict() should cope with, that's fine. If you want someone else to go do the work for you, that's less fine, but hey, someone might think it's an interesting thing to do. If you want dict() to cope with arbitrary objects (which you still appear to), you need to say how.
I still think this is an unhelpful starting place. I would actually be hesitant to use an API that just said "oh, throw anything at me, I'll cope." I'd much rather use something that told me what format it expected data in, what I need to conform to, and do any conversion myself. With half an ounce of luck I'll be able to arrange to get the data in the right format in the first place and avoid a whole unnecessary layer of processing.
For user-defined protocols, we simply leave HOW to dict-ify to the caller via the factory_dict argument (if it is provided).
I am minded of the Larson cartoon, "...and now a miracle occurs..." This is the first time you've mentioned this argument (to dict(), presumably?).
Right. So what you want is to add a keyword argument to dict() such that dict(obj, factory_dict=dict_me_harder) calls dict_me_harder(obj) if obj is not a type dict() knows how to deal with natively, expecting it to return a new dict or raise an exception. Is that right? -- Rhodri James *-* Kynesim Ltd
Here I was talking about the "mapping protocol" (define keys() and getitem()) but as you say, this essentially turns your object INTO a mapping. That isn't the same as some dict-making API (well I guess, unless you include returning two-tuples from iter). That was bad phrasing on my part.
If all you want
is to add dataclasses and namedtuples to the list of things dict() copes
At this point I am really exploring the problem and potential solutions. Your responses have been helpful. I do not have a list of obviously mapping-like things that eschew the mapping protocol, other than NT and dclses. It would be helpful to know if more of these exist. I'd be happy to try and look for them, but I don't really know how I'd do that. You bring up a good point-- dict() likely makes more sense for this than expanding the dataclasses.asdict(). Looking at your menu of choices (which I think hit on most of the potential solutions), I would change my initial approach to the problem and suggest instead that two ideas should be considered: - add NamedTuple._asdict() and dataclasses.asdict() to the list of things dict() copes with - add an alt constructor that behaves just like the dict() constructor, but "dicts harder", as you put it But I see two problems with doing it using the dict() constructor: - typing.NamedTuple and @dataclass classes can also define their own keys() and getitem()-- the behavior of which might diverge from their respective asdict() methods. So dict() would need to decide whether nt._asdict() and dataclasses.asdict() take precedence over the mapping protocol (I would suggest that IF the mapping protocol exists on an object, it would take precedence). - I am sure there is lots of code out there currently depending on errors occuring when dict() is applied to a NT. So of the two options, I think an alt constructor is far superior. The alt constructor would still need to decide the first precedence problem though. I am minded of the Larson cartoon, "...and now a miracle occurs..."
This is the first time you've mentioned this argument (to dict(), presumably?).
I would actually be hesitant to use an API that just said "oh, throw anything at me, I'll cope." Well my initial idea (now reconsidered) was to change the behavior of dataclasses.asdict(), which does already have a dict_factory kwarg. I was referring to that asdict() argument. However for me, adding a kwarg to dict() is not on the table; dict() makes use of **kwargs and reserving one for this purpose is surely a bad idea. That last comment makes sense to me, but the alternative- adding handling of NT and dclses to dict()- seems bad because of backward compatibility problem I mentioned above. Therefore because of the constructor **kwargs problem, and because I agree miracles are bad things in programming :), and because you are right that it is better for the user to present the data in an expected format, adopting a dict_factory kwarg is probably not worth bothering with. I'm to happy to abandon that.
On 09/05/2019 14:42, Ricky Teachey wrote:
The only thing I can think of is going through the standard library documentation (or the code if you prefer) and seeing what leaps out as dict-like but not actually a mapping. That sounds like a lot of work to me :-( -- Rhodri James *-* Kynesim Ltd
Another difficulty I just thought of: - a NT class- and a dcls, for that matter- could also be an iterable two-tuples that would currently get dict-ified using iteration. So a precedence rule- does iter() win? Or the asdict functions?- would be needed for this, too. The answer to that seems a lot less obvious to me (compared to the mapping protocol), since iterating k-v pairs is relying on a bit of a side effect, but that side effect has probably existed since the dict was created.
Frankly, I don't think an "as dict" protocol is worth it, but if others did think so, we could: define a new dunder: __as_mapping__ (I'm using "mapping", rather than "dict", to allow duck typing) Then the built-in dict() constructor would: check for an __as_mapping__ dunder: -- if there, it would call it and expect to get a mapping back that it could turn into a dict. -- if not, it would go through its current process ( you could perhaps specify a subset of the mapping interface -- maybe all you'd need is .items(), for instance -- so it would actually return an iterator of key-value pairs. after all, dict() can already consume that.) Meanwhile, any other code that wants to could also use the __as_mapping__ dunder to be able to work with an object as though it were a mapping, even if that isn't its usual interface. As far as I can see, the only downside of this is that it would means a new standard dunder, which is a substantial downside, as (I think) it essentially adds overhead in all sorts of places, plus ads "One more protocol" to a language already ripe with protocols. At least that's the message I got when I suggested adding a dunder for a sorting key method. -CHB On Thu, May 9, 2019 at 10:13 AM Ricky Teachey <ricky@teachey.org> wrote:
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
I like the __as_mapping__ idea a lot since it leaves it up to the user to determine how dict() does dict-making. But I understand how the overhead from adding a single function call could be very significant to something that is used as often as dict(). What about pairing a __mapping__ method with a mapping() function instead? mapping() would not need to be a keyword; it might make sense as part of, perhaps, the types library. types.mapping() would be significantly slower than dict() only sometimes-- and resource critical code could still rely on dict(). But it would also be more forgiving. And possibly even more useful (for the reasons I've tried to do a good job of outlining).
On 10/05/2019 21:22, Ricky Teachey wrote:
I come back to the point of what the use cases for this are. So far you have presented a situation in which you have made your life difficult in order to make other people's lives easier (though I'd contend it doesn't actually do that). If this is a common enough occurrence, that would justify adding the (relatively small and self-contained) functionality to dict(). I personally doubt it, but I'm ready to be proved wrong.
It would still require other types to add and maintain __mapping__ dunders. While the bar for this is lower than adding functionality to dict(), I'm still not sure you've got over it. -- Rhodri James *-* Kynesim Ltd
It would still require other types to add and maintain __mapping__ dunders.
Sure — but that is inherent in this whole idea — nothing other than the type object itself could possibly know how to dictify itself. Though yeah, that’s the key question— how often is this useful??? -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
Sure — but that is inherent in this whole idea — nothing other than the type object itself could possibly know how to dictify itself.
This seems to have an obvious solution: object.__as_mapping__() could simply call dict(self). Anything that uses the mapping protocol could then be mappified. Though yeah, that’s the key question— how often is this useful???
I'll try to make a quick case it is. Dataclasses have existed for barely a year. I don't know how widely used they are yet, by I sure use the beschtick out of them. The API is very nice and I fully expect them to be wildly successful. But the dclses API encourages the use of the asdict function rather than the mapping protocol-- if they become successful and widely used, it will become more and more inconvenient to have to test for multiple mapping APIs. And this isn't the first alt mapping API. NT is already incredibly successful (to the degree it has been an intention of the core devs to use it more and more in core python libraries), and as such there is a ton of if isinstance(obj,NT) obj._asdict() else dict(obj) code out there. Another dictish type in the std lib is types.SimpleNamespace, which is typically dictified (or rather, re-dictified) via yet another API, dict(obj.__dict__). The more we pile these APIs upon APIs, the more ridiculous it gets to create a flexible API of your own that allows the user to choose what kind of structure representation THEY want to provide.
Python allows for breaking out from a loop through 'break' and 'continue' keywords. It would be nice if it was possible to break many loop levels using one command. I propose two constructions for that: break break break break break break ... continue break continue break break continue ... And so on. Example: for i in range(10): for j in range(10): if i == 2 and j == 3: break break for i in range(10): for j in range(10): if i == 2 and j == 3: break continue if i == 7: break break Breaking out from many loops at once is a common practice, currently implemented through boolean testing and exceptions. This proposal would make it cleaner.
Dear Haael With python you can write for i, j in something (): # loop body This would cover both your examples. For the first, use itertools.product. For the second, write a custom iterator. -- Jonathan On Sat, 11 May 2019, 18:22 haael, <haael@interia.pl> wrote:
On Sat, 11 May 2019 at 18:22, haael <haael@interia.pl> wrote:
Breaking out from many loops at once is a common practice
Do you have evidence for this? In my experience, it's very rare (although I concede it would be *occasionally* convenient). In most cases where I'd consider breaking out of multiple levels of nested loop, I've tended to find that it's probably more readable to refactor the code to use a generator, or something like that. Paul
I don't love the 'break break' syntax idea. But I find myself wanting to break out of nested loops quite often. Very rarely could this be transformed to an 'itertools.permutation' de-nesting, albeit perhaps more often than I actually do that. My usual kludge is a sentinel 'break_outer' variable that gets set back and forth between True and False working the loops. Usually work a name that descriptive of the actual domain of the code. On Sat, May 11, 2019, 1:40 PM Paul Moore <p.f.moore@gmail.com> wrote:
On 11 May 2019, at 23:20, David Mertz <mertz@gnosis.cx> wrote:
I don't love the 'break break' syntax idea. But I find myself wanting to break out of nested loops quite often. Very rarely could this be transformed to an 'itertools.permutation' de-nesting, albeit perhaps more often than I actually do that.
Do you have an example of such loops ?
judicious use of the for loop "else" may help here, too -- I know I often forget it's there. -CHB On Sat, May 11, 2019 at 2:26 PM Adrien Ricocotam <ricocotam@gmail.com> wrote:
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
On Sun, May 12, 2019 at 3:45 AM Christopher Barker <pythonchb@gmail.com> wrote:
judicious use of the for loop "else" may help here, too -- I know I often
forget it's there. yeah, I often do this when I want to break twice but this method only supports breaking twice at once but this doesn't support break once and break twice semantics simultaneously. ``` for x in range(n): for y in range(m): break else: continue break ``` On Sun, May 12, 2019 at 3:45 AM Christopher Barker <pythonchb@gmail.com> wrote:
-- Regards Srinivas Devaki Software Developer at Zomato, New Delhi Phone: +91 9491 383 249 Telegram: @eightnoteight
On Sat, May 11, 2019 at 1:22 PM haael <haael@interia.pl> wrote:
I'm surprised no one has yet mentioned splitting off the loops into a function and using "return" to break out of those loops. If you nest the function in the spot where the loops would be and use nonlocal statements, you can even modify variables in the surrounding code (counters, etc.) or leave values behind. If you're opposed to a function and performance is not critical [1], you can also raise a custom "exception" at the multi-break spot and handle it at the appropriate place in the outer code. This is useful if you need to break out of a function plus additional nesting layers. [1] I mention performance here because if I recall correctly, exception handling is quite expensive and slow in Python.
On 5/11/2019 1:20 PM, haael wrote:
Breaking out from many loops at once is a common practice, currently implemented through boolean testing and exceptions.
Python exceptions generalize both 'return' and 'break' as methods for escaping nested contexts and are intended for generalized flow control.
This proposal would make it cleaner.
I actually think that the general method is 'cleaner', be separating 'leave here' and 'arrive here'. This proposal would mean typing fewer words in the special case where there are nested loops within one function. The cost is a) more syntax to learn and b) tightly tying together the two loops more tightly than might be necessary, and thereby inhibiting re-factoring. In the examples above, where the loops are inherently tied together, the two loops can, as noted by others, be reduced to one. If they are not inherently tied together, one might want to reduce the inner loop to a comprehension or wrap it in a new function so it can also be used elsewhere. If the inner loop raises, wrapping or unwrapping it, as needed for reusability or efficiency, does not affect the logic. -- Terry Jan Reedy
Terry reminds me of a common case I encounter that cannot be transformed into a loop over itertools.product(). E.g. for main in stuff: if thing_about(main): for detail in more_stuff(stuff, main): if seen_enough(detail): # break to outer somehow else: for detail in different_stuff(): if seen_enough(detail): # break to outer somehow On Sat, May 11, 2019, 6:58 PM Terry Reedy <tjreedy@udel.edu> wrote:
On Sun, May 12, 2019 at 12:43 PM David Mertz <mertz@gnosis.cx> wrote:
For that kind of loop, there's a different problem, which is the duplication. So I would start by rewriting that as: for main in stuff: if thing_about(main): details = more_stuff(stuff, main) else: details = different_stuff() for detail in details: if seen_enough(detail): ... I'm not sure what you mean by "break to outer somehow", though, so I don't know what you're actually needing here. Concrete examples would help. ChrisA
I remembered, there is a "labeled break" in Perl. https://perldoc.perl.org/functions/last.html On Sun, May 12, 2019 at 10:50 AM Chris Angelico <rosuav@gmail.com> wrote:
Ok, sure. But what if different seen_enough() conditions are in the two inner loops? By "break to outer" ... I mean, don't get any more 'main' from 'stuff'. I meant to dig through some real code, but forgot because I was writing code for work. This example on my tablet is the general pattern I often need though. And obviously I've solved it one way or another hundreds of times. I think a "labelled break" would be nice though. It might have even been a past PEP. On Sat, May 11, 2019, 10:51 PM Chris Angelico <rosuav@gmail.com> wrote:
On Sun, May 12, 2019 at 1:16 PM David Mertz <mertz@gnosis.cx> wrote:
Ok, sure. But what if different seen_enough() conditions are in the two inner loops? By "break to outer" ... I mean, don't get any more 'main' from 'stuff'. I meant to dig through some real code, but forgot because I was writing code for work. This example on my tablet is the general pattern I often need though. And obviously I've solved it one way or another hundreds of times.
I think a "labelled break" would be nice though. It might have even been a past PEP.
This is why concrete examples are much easier to discuss, heh. In any case, it's often possible to either (a) redefine the inner loop as some sort of container operation, or (b) redefine the outer operation as a function, and use "return". So it would be helpful to have an example that CAN'T be rewritten in one of those ways, to use as a compelling example. ChrisA
I'll try to find a concrete one tomorrow. I did this recently, but I might have factored it away somehow. The real life code has lots of extraneous parts that need to be removed or simplified though. Both because they aren't FLOSS, and because the details would muddy things. "Trimmed from real" won't look much different than "invented from scratch". And yes, OF COURSE everything CAN be rewritten. But that's true in regards to *every* proposed addition. The sentinel break_outer variable is often the was to do it without really big restructuring (but with a handful of ugly bookkeeping lines instead). On Sat, May 11, 2019, 11:22 PM Chris Angelico <rosuav@gmail.com> wrote:
On Sun, May 12, 2019 at 1:30 PM David Mertz <mertz@gnosis.cx> wrote:
I'll try to find a concrete one tomorrow. I did this recently, but I might have factored it away somehow. The real life code has lots of extraneous parts that need to be removed or simplified though. Both because they aren't FLOSS, and because the details would muddy things. "Trimmed from real" won't look much different than "invented from scratch".
And yes, OF COURSE everything CAN be rewritten. But that's true in regards to *every* proposed addition. The sentinel break_outer variable is often the was to do it without really big restructuring (but with a handful of ugly bookkeeping lines instead).
Yes - that's exactly the sort of thing that makes a great example. Show the way you'd LIKE to write it, and then show how you have to write it using current tools. If the latter is really ugly, requires finicky bookkeeping, or is error-prone, it's a good argument in favour of the new feature. ChrisA
To be clear in this thread, I don't think I'm really ADVOCATING for a multi-level break. My comments are simply noting that I personally fairly often encounter the situation where they would be useful. At the same time, I worry about Python gaining sometimes-useful features that complicate the overall language and bring it closer to Perl's "everyone can write in their own style." So to answer Chris' question... well, i did find something recent, but it has a ton of other extraneous stuff. But I've simplified to something that isn't that much different from my example from my tablet yesterday, but is fairly realistic still. I have an existing function that looks roughly like this: def find_needle_in_haystacks(): found_needle = False for haystack in glob.glob('path/to/stuff/*'): fh = open(fname) header = fh.readline() if get_format(header) == 'foo': for line in fh: status = process_foo(line) if status = -1: found_needle = True break elif get_format(header) == 'bar': for line in fh: status = process_bar(line) if status = -1: found_needle = True break if found_needle: break If I had a "labelled break" feature, I might rewrite it something like the following (picking convenient syntax that I'm not per-se advocating as best, even if the concept is adopted): # Hypothetical future labelled break: def find_needle_in_haystacks(): for haystack in glob.glob('path/to/stuff/*') label HAYSTACKS: fh = open(fname) header = fh.readline() if get_format(header) == 'foo': for line in fh: if process_foo(line) == -1: break HAYSTACKS elif get_format(header) == 'bar': for line in fh: if process_bar(line) == -1: break HAYSTACKS In any case, it's often possible to either (a) redefine the inner loop
-- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th.
On Sun, 12 May 2019 at 18:26, David Mertz <mertz@gnosis.cx> wrote:
Nice. Suggestion: 1. replace "label HAYSTACKS" with "as HAYSTACKS", if possible, so no new keyword is introduced; 2. replace "break HAYSTACKS" with "break from HAYSTACKS"
-- Gustavo J. A. M. Carneiro Gambit Research "The universe is always one step beyond logic." -- Frank Herbert
On Sun, May 12, 2019, 3:33 PM Gustavo Carneiro <gjcarneiro@gmail.com> wrote:
I thought of 'as' initially, and it reads well as English. But it felt to me like the meaning was too different from the other meanings of 'as' in Python. I might be persuaded otherwise. Likewise, 'from' isn't really very similar to other existing uses.
On Sun, 12 May 2019 at 21:06, David Mertz <mertz@gnosis.cx> wrote:
I thought of 'as' initially, and it reads well as English. But it felt to me like the meaning was too different from the other meanings of 'as' in Python. I might be persuaded otherwise.
If you think in terms of "named loops" rather than "labels", "as" makes a lot more sense. Having said that, I don't think this change is worth it, so debates about naming are mostly just theoretical to me :-) Paul
On Sun, May 12, 2019, 5:36 PM Paul Moore <p.f.moore@gmail.com> wrote:
Every name created with 'as' currently is a regular Python object, with a memory address, and some methods, that can participate in comparison (at least equality), that has a type, can be passed as a function argument, etc. Actually, all that is true of EVERY name in Python other than keywords. It seems like none of this would be true of a loop name/label. Hence my first thought that a different way of introducing the word is more clear... Maybe we can just prefix lines with labels instead, and add a 'goto' command :-)
How about doing it pythonic way then, "everything is a hack on a symbol table". Let normal loops remain normal loops. Let's introduce a special construction:
for x in iterator as loop_control_object: loop_body(loop_control_object)
The iterator in the loop would be wrapped inside another iterator that would yield everything as the original iterator, but it would also be controllable through loop_control_object. Sample implementation:
The control object could have methods that allow sophisticated loop control, such as breaking, continuing, injecting exceptions, counting iterations, skipping etc. The keywords 'break loop_control_object' and 'continue loop_control_object' could be simply implemented as:
loop_control_object.do_break()
I thought of "we could return immediately" shortly after I posted it. Yes, that's true in the example I wrote. But it's easy enough to vary it so that something happens after the loop as well. I also noticed that for the simple version, it might be slightly shorter to put the conditional inside the loop rather than to decide between loops. There are lots of ways to work around the absence of a labelled break. Many times putting the inner loop in a comprehension can simplify things too, for example (that's roughly what my actual code was refactored to, mostly not just because of the break issue). On Sun, May 12, 2019, 3:38 PM Chris Angelico <rosuav@gmail.com> wrote:
The concrete example I was working on when I started to miss double break. This is an implementation of polynomial long division in Galois field. Almost unmodified. With outer break, I would't need to use the `running` variable. In fact, for mathematical clarity, I would like to put a test in the outer loop, like: `while dividend`. It would make clear what this algorithm is actually doing. Sure, it could be refactored, but it would lose the relative simplicity. For such operations it is sometimes important to remain close to the raw mathematical notation. def __divmod__(self, other): dividend = self.canonical() try: divisor = other.canonical() except AttributeError: return NotImplemented if not dividend: return Field.zero(self.size), Field.zero(self.size) try: d = next(iter(divisor)) # leading term of the divisor except TypeError: d = self.const(divisor) do = self.__monomial_order(d) result = Field.zero(self.size) running = True while running: # outer loop, retry the changing `dividend` for x in dividend: # yield all monomials from this polynomial if self.__monomial_order(x) < do: # encountered monomial of too small order, finish running = False break # exit outer loop try: c = self.__monomial_division(x, d) # may fail assert c # sanity check result += c dividend -= c * divisor dividend = dividend.canonical() # if `dividend` nonzero, try again # if `dividend` is zero, finish running = bool(dividend) break except ArithmeticError: # monomial division failed, try next one continue # inner loop else: # `dividend` exhausted, finish running = False pass if not hasattr(result, 'operator'): result = self.const(result) return result.canonical(), dividend.canonical()
If I may propose `break n` as a replacement for the original message's `break break ... break`, where n>0 is the number of contiguous loops to break out of and `break 1` is synonymous with `break`. Seems easier on my eyes/noggin than counting out the individual `break` statements. Eli On Sat, May 11, 2019, 10:21 AM haael <haael@interia.pl> wrote:
On Sun, May 12, 2019 at 01:36:28AM -0700, Elias Tarhini <eltrhn@gmail.com> wrote:
This is very much error-prone because on any refactoring (increasing or decreasing the number of loop levels) one must increase/decrease all numbers in internal loops. Labels are at least stable.
Eli
Oleg. -- Oleg Broytman https://phdru.name/ phd@phdru.name Programmers don't die, they just GOSUB without RETURN.
On Sun, May 12, 2019 at 11:16:21AM +0200, Oleg Broytman wrote:
Indeed. Go has labels for loops: Label: for { ... break Label } as does Rust: 'label: while condition: { ... break label } We can't use the same syntax in Python, but we might write it like this: @label while condition: block break label and similar with for loops: @outer for x in range(100): @inner for y in range(3): if x == y == 1: break # like ``break inner`` if x == y == 2: break outer print(x, y) # break inner jumps to here # break outer jumps to here -- Steven
What if labels are placed in comments beside the loops, like how type is, at least, being used by analyzers? for x in range(100): # label: outer for y in range(3): # label: inner On Sun, May 12, 2019 at 5:45 PM Steven D'Aprano <steve@pearwood.info> wrote:
On 2019-05-12 10:44, Steven D'Aprano wrote:
@ reminds me too much of decorators. How about this: <label> while condition: block break label <outer> for x in range(100): <inner> for y in range(3): if x == y == 1: break # like ``break inner`` if x == y == 2: break outer print(x, y) # break inner jumps to here # break outer jumps to here
On 11/05/2019 02:46, Christopher Barker wrote:
Though yeah, that’s the key question— how often is this useful???
I have wanted something similar to this in Jython. Java supplies a battery of containers (with different performance and concurrency characteristics) and we monkepatch onto them the methods of list, set or dict according to their interface as java.util.List, Set or Map. This works "slightly badly", because sometimes those Python types have methods with the same name as one in the underlying container, but a different meaning. E.g. list.pop() disgrees about which end. I'd much rather have some kind of explicit wrapping or cast into their Python type: I had imagined the wrapped object would identify and behave as exactly that Python type (or ABC), however, it would be a *proxy* to the original. Thus one obtains a "view" onto the original collection, or looked at another way, a way to choose the implementation behind a particular Python dict, set or list. Several alternative ways of expressing this come to mind, but .asdict() is one. It's not always clear in this discussion whether a proxy is envisaged (where a put() goes through to the original object, if allowed) or just a way of constructing and filling a new dict. The latter would not fit my imagined use. Jeff Jeff Allen
On Sat, May 11, 2019 at 4:06 AM Jeff
The ABCs are already defined — if you want to make a wrapper proxy object, you implement the ABC you want to emulate. Now that I’m thinking about it more, this really covers most use cases: if you want your object to be converted to a dict by passing it to dict(), then implement the iteration protocol, returning key-value pairs that you want. This would work fine for dataclasses, and SimpleNamespaces, if one wanted to implement that. The only problem here is if you want to use the standard iteration protocol for something different -- I can't image what that would be. Sure, on the general case, maybe, but for a class that has a "natural" use case for dictifying, how else would you want to iterate over it?? -CHB
On 2019-05-11 12:50, Christopher Barker wrote:
One example would be. . . dicts. Calling dict on a dict gives you a dict, but iterating over the dict just gives you the keys. Certain other mapping types (such as Pandas Series and DataFrames) also have this behavior where iterating over the mapping gives you only the keys. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown
On Sat, May 11, 2019 at 1:38 PM Brendan Barnwell <brenbarn@brenbarn.net> wrote:
well, there is that ... :-) And that IS the mapping protocol. I guess i was thinking that you had an object that was NOT a Mapping already. If it IS a Mapping, then presumably it already can be passed into the dict constructor, yes? So if you want your class "dictable", then you can implement either the Mapping ABC, or the iteration protocol. I"m having an even harder time imagining why you would want a Mapping interface that isn't the same as what you'd want turned into a dict. So why again do we yet another way to do it? Anyway, I was playing with this, and write a prototype decorator that makes a datacalss "dictable" -- that it, it can be passed in to the dict constructor to make a dict. This is done by providing an iterator that returns (key, value) pairs. The code is in this git rep, if anyone wants to comment on it or improve, it, or... https://github.com/PythonCHB/PythonTopics/blob/master/Sources/code/dict_maki... (also here for your reading pleasure) I'm not sure how the dict constructor decides an input is a mapping or an iterable -- maybe I'll play with that now. -CHB #!/usr/bin/env python """ A prototype of a decorator that adds an iterator to a dataclass so it can be passed in to the dict() constructor to make a dict. If this is thought to be useful it could be added to the dataclass decorator itself, to give all decorators this functionality. """ from dataclasses import dataclass class DataClassIterator: """ Iterator for dataclasses This used the class' __dataclass_fields__ dict to iterate through the fields and their values """ def __init__(self, dataclass_instance): self.dataclass_instance = dataclass_instance self.keys_iter = iter(dataclass_instance.__dataclass_fields__.keys()) def __iter__(self): return self def __next__(self): key = next(self.keys_iter) return (key, getattr(self.dataclass_instance, key)) def _dunder_iter(self): """ function used as the __iter__ method in the dictable_dataclass decorator """ return DataClassIterator(self) def dictable_dataclass(the_dataclass): """ class decorator for making a dataclass iterable in a way that is compatible with the dict() constructor """ the_dataclass.__iter__ = _dunder_iter return the_dataclass # Example from the dataclass docs: @dictable_dataclass @dataclass class InventoryItem: '''Class for keeping track of an item in inventory.''' name: str unit_price: float quantity_on_hand: int = 0 def total_cost(self) -> float: return self.unit_price * self.quantity_on_hand if __name__ == "__main__": # try it out: inv_item = InventoryItem("sneakers", 50.0, 20) print("an InventoryItem:\n", inv_item) print() print("And the dict you can make from it:") print(dict(inv_item))
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
OK, so some more exploration: Other may already know this, but it seems that the dict constructor essentially checks if the passed in object is a Mapping ABC: if it is, it iterates on teh keys and used __getitem__ to get the values. if it is not, it iterates over key, value pairs. So if you want your custom class to present the Mapping interface, and be able to be passed in to dict(), then you shoudl subclass from collections.Mapping.ABC If you do not want your class to present the Mapping interface, then your __Iter__ should return an iterator that returns key.value pairs. Here is some code that explores that with a custom Mapping class. https://github.com/PythonCHB/PythonTopics/blob/master/Sources/code/dict_maki... (also below) The trick here is that with, e.g. the dataclasses decorator, I'm not sure how to make it derive from the Mapping ABC -- I'm sure it's doable, but I haven't dug into that yet. Personally, I'm a big fan of duck typing -- so rather than check for the Mapping ABC, I think it should check for .items() first, and use that if it's there, and then use the regular iterator otherwise. Then all you'd have to do to make your class "dictable" would be to provide an items() iterator. But again -- there are two ways to make your classes dictable -- why do we need a third? -CHB Sample code: """ an exploration of how the dict constructor knows whether it is working with a MApping of a general iterable. It looks like the code is something like this: if isinstance(input, collections.abc.Mapping): self.update = {key: input[key] key in input} else: self.update = {key: val for key, val in input} So if your custom class present themapping interface -- that is, iterating over it returns the keys, than you want to be a MApping ABC subclass.ABC But if your custom class is not a Mapping, then you want __iter__ to return an iterator over teh key, value pairs. """ from collections.abc import Mapping def test_iter(instance): yield ("this") yield ("that") class DictTest(Mapping): def __init__(self, this="this", that="that"): self.this = this self.that = that def __iter__(self): print("iter called") return test_iter(self) def __getitem__(self, key): return getattr(self, key) def __len__(self): print("__len__ called") if __name__ == "__main__": dt = DictTest() print(dict(dt)) dt = DictTest(this=45, that=654) print(dict(dt)) On Sat, May 11, 2019 at 1:51 PM Christopher Barker <pythonchb@gmail.com> wrote:
-- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
It’s a design goal of dataclasses to not be iterable. https://www.python.org/dev/peps/pep-0557/#why-not-just-use-namedtuple Eric
Using iter() for the sole purpose of being able to shoehorn your user class into being dict-cast-able is, IMO, an abuse of iter. If your class makes logical sense as an iterable of key/value pairs, then for petesake, just use the mapping protocol (implement getitem and keys). iter() is for constructing sequences, not mappings. And if the class DOESN'T make logical sense as an interable of key/value pairs, it still isn't time to abuse iter() for the convenience of making dicts. Just make an asdict() method. The above, I believe, is great API advice that I didn't come up with myself-- and it's so great that it has been followed by both NT and dclses. But the downside is, we now have the inconvenience of multiple dict-making APIs.
On Sun, May 12, 2019 at 10:14 AM Ricky Teachey <ricky@teachey.org> wrote:
Using iter() for the sole purpose of being able to shoehorn your user class into being dict-cast-able is, IMO, an abuse of iter.
If your class makes logical sense as an iterable of key/value pairs, then for petesake, just use the mapping protocol (implement getitem and keys). iter() is for constructing sequences, not mappings.
iter() is for collections of all sorts; a sequence is specifically an ordered and indexed collection, but you can iterate over many things, including sets - and mappings. There's nothing wrong with iterable mappings, and in fact, I would be surprised (and not in a good way) by any that wasn't. ChrisA
On Sat, May 11, 2019 at 3:26 PM Eric V. Smith <eric@trueblade.com> wrote:
It’s a design goal of dataclasses to not be iterable.
https://www.python.org/dev/peps/pep-0557/#why-not-just-use-namedtuple
you would know, but that reference talks about why they are not the same as NamedTuple. if dataclasses *were*iterable, they almost certainly wouldn't iterate over the values alone. And NamesTuple was designed to BE tuples -- that is a drop in replacement for tuples -- so they have the features they have partially for that reason. And my toy code actually adds another decorator to make dataclasses iterable, so it would be a completely optional feature. All that being said, I'm not actually advocating doing this with dataclasses -- I'm simply making the point that there are currretly two ways to make custom class able to be passed in to the dict constructor -- do we need another? -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
That section mentions why they’re not iterable. Search on “iterable”.
if dataclasses *were*iterable, they almost certainly wouldn't iterate over the values alone.
That wouldn’t make a difference. The given NT example would still be a problem.
And NamesTuple was designed to BE tuples -- that is a drop in replacement for tuples -- so they have the features they have partially for that reason.
And my toy code actually adds another decorator to make dataclasses iterable, so it would be a completely optional feature.
That approach of adding support for iterability makes sense to me. I’m contemplating adding a “dataclasses_tools” package for some other tools I have. But I’m not sure how this fits in to the asdict discussion. Eric
On Sat, May 11, 2019 at 8:23 PM Eric V. Smith <eric@trueblade.com> wrote:
Sure, but in the context of the difference with NamedTuple: """ Instances are always iterable, which can make it difficult to add fields. If a library defines: Time = namedtuple('Time', ['hour', 'minute']) def get_time(): return Time(12, 0) Then if a user uses this code as: hour, minute = get_time() then it would not be possible to add a second field to Time without breaking the user's code. """ which is a great argument for why you wouldn't want dataclasses to iterate on the *values* of the fields, which is what NamedTuple does. In fact, beyond iteration, NamedTuple, IS a tuple, so: - the number of "fields" matters, - the order of the "fields" matters, - it's the values that really matter -- the field names are incidental. None of this (intentionally) applies to dataclasses - which is what i took as the point of that section of the PEP. But I see dataclasses as being more analogous to a dict than a tuple: - The field names matter - The order of the fields does not matter (the current state of the dict implementation not withstanding) So *if* dataclasses were to be iterable, then I'd think they should either implement the Mapping protocol (which would make them, similar to NamedTuple, a drop-in replacement for a dict, or iterate over (field_name, value) pairs. which would not introduce the issues staed in teh PEP with the iterability of NamedTuple. Note: I am NOT advocating for making dataclasses a Mapping -- I believe in a clear separation between "data" and "code" -- dicts are suitable for data, and dataclasses are suitable for code. Given PYthon's dynamic nature, this isn't always a clear boundary, but I think it's a better not to purposely make it even less clear. Others may have a different take on this -- it could be kind of a cool way to make a dict with extra functionality, but I think that's ill-advised.
you know, I'm embarrassed to say that I read through a lot of this thread wondering what the heck an NT is ;-) -- but now I got it. but I don't think the NT example is relevant here -- it's an issue with NT, because NTs are designed to be, and used as drop in replacements of tuples, and a lot of code out there does use tuples for "unpacking" assignments. But I'm going to hazard a guess that it's pretty rare for folks to do: def get_time(): ... return {"hour", 3, "minute", 5} hour, minute = get_time() and: result = get_time() hour = result['hour'] minute = results["minute"] wouldn't break, now would: time_dict = dict(get_time()) Anyway, we need dataclasses AND NamedTuples because dataclasses down't behave like a tuple -- but that doesn't mean that they couldn't behave more like a dict in some ways.
I like that -- good idea. If any of my toy code is useful, please feel free to make use of it.
But I’m not sure how this fits in to the asdict discussion.
Well, it fits in in the sense that the question at hand is: Is there a need for a standard protocol for making dicts out of arbitrary classes? datacalsses were used as an example use-case, as datacalsses do naturally map to dicts. My point is that there are already two protocols for making an arbitrary classes into a dict, so I don't think we need another one. Even though those two are overloading other use cases: the Mapping ABC and iteration. This does mean that there would be no way to shoehorn NamedTuples into this -- as they already use the iteration protocol in a way that is incompatible with dict creation. So yeah, maybe a use case there. But the other option is to use a convention, rather than an official protocol -- if an object has an "asdict()" method, then that method returns a dict. pretty simple. If you were dealing with arbitrary objects, you'd have to write code like: try: a_dict = an_object.asdict() except (AttributeError, Type Error): a_dict = something_else but you'd need to do that anyway, as you couldn't count on every type being conformant with the dict() constructor anyway. So I'm still trying to see the use cases for having wanting to turn arbitrary objects into dicts. As for NamedTuple -- it does have _asdict: In [122]: p._asdict() Out[122]: OrderedDict([('x', 4), ('y', 5)]) (sidenote --now that dicts preserve order, maybe NamedTuple could return a regular dict) So I see the OP's point -- and it's a leading dunder name because: """ To prevent conflicts with field names, the method and attribute names start with an underscore. """ which I suppose means that if there is an informal protocol, it should also use that leading underscore. -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython
participants (22)
-
Adrien Ricocotam -
Brendan Barnwell -
Chris Angelico -
Christopher Barker -
David Mertz -
Elias Tarhini -
Eric V. Smith -
Gustavo Carneiro -
haael -
James Lu -
Jeff Allen -
Jonathan Fine -
Jonathan Goble -
MRAB -
Oleg Broytman -
Paul Moore -
Rhodri James -
Ricky Teachey -
Ronie Martinez -
srinivas devaki -
Steven D'Aprano -
Terry Reedy