From brittocj at gmail.com Wed May 1 22:46:18 2019 From: brittocj at gmail.com (Britto .) Date: Thu, 2 May 2019 08:16:18 +0530 Subject: [Python-ideas] Populating a timetable with subjects Message-ID: Hi All, We have to populate a timetable with subjects. What would be the best approach? In our sample timetable there are 25 hours (keys), all the FH value should be replaced with random subjects as many as their corresponding hour. timetable = [{'A1': "FH", 'B1': "FH", 'C1': "FH", 'D1': "FH", 'E1': "FH"}, {'A2': "FH", 'B2': "FH", 'C2': "FH", 'D2': "FH", 'E2': "FH"}, {'A3': "FH", 'B3': "FH", 'C3': "FH", 'D3': "FH", 'E3': "FH"}, {'A4': "FH", 'B4': "FH", 'C4': "FH", 'D4': "FH", 'E4': "FH"}, {'A5': "FH", 'B5': "FH", 'C5': "FH", 'D5': "FH", 'E5': "FH"}] subjects_required_hours = {"S1": 4, "S2": 3, "S3": 5, "S4": 3, "S5": 5, "S6": 5} Regards Britto -------------- next part -------------- An HTML attachment was scrubbed... URL: From brittocj at gmail.com Wed May 1 23:00:37 2019 From: brittocj at gmail.com (Britto .) Date: Thu, 2 May 2019 08:30:37 +0530 Subject: [Python-ideas] Populating a timetable with subjects In-Reply-To: <20190502025934.GB5010@ando.pearwood.info> References: <20190502025934.GB5010@ando.pearwood.info> Message-ID: Cool, thanks! Regards Britto On Thu, 2 May 2019 at 08:29, Steven D'Aprano wrote: > Hi Britto, and welcome. > > This mailing list is for proposing new features for the Python language, > not a help desk or for asking for help with what looks like a homework > question. You might like to try the "python-list" mailing list or > Reddit's learnpython subreddit, or perhaps Stackoverflow. > > All the best, > > > > -- > Steven > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Wed May 1 22:59:35 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 2 May 2019 12:59:35 +1000 Subject: [Python-ideas] Populating a timetable with subjects In-Reply-To: References: Message-ID: <20190502025934.GB5010@ando.pearwood.info> Hi Britto, and welcome. This mailing list is for proposing new features for the Python language, not a help desk or for asking for help with what looks like a homework question. You might like to try the "python-list" mailing list or Reddit's learnpython subreddit, or perhaps Stackoverflow. All the best, -- Steven From eric at trueblade.com Thu May 2 08:24:11 2019 From: eric at trueblade.com (Eric V. Smith) Date: Thu, 2 May 2019 08:24:11 -0400 Subject: [Python-ideas] f-string "debug" conversion In-Reply-To: <3473f538-7683-28c0-c45e-3e0776bfd7ed@trueblade.com> References: <3473f538-7683-28c0-c45e-3e0776bfd7ed@trueblade.com> Message-ID: <48de2771-fa10-ae8e-b188-8fc8509109a2@trueblade.com> See https://bugs.python.org/issue36774. On 10/2/18 8:27 PM, Eric V. Smith wrote: > This idea was proposed to me at the core sprints last month by Larry > Hastings. I've discussed it with a few people, who seem generally > positive about it, and we've tweaked it a little bit. I've spent some > time implementing it, and I think it's doable. I thought I'd post it > here for any additional feedback. > > Here?s the idea: for f-strings, we add a !d conversion operator, which > is superficially similar to !s, !r, and !a. The meaning of !d is: > produce the text of the expression (not its value!), followed by an > equal sign, followed by the repr of the value of the expression. So: > > value = 10 > s = 'a string!' > print(f'{value!d}') > print(f'next: {value+1!d}') > print(f'{s!d}') > > produces: > > value=10 > next: value+1=11 > s='a string!' > > I?m not proposing this for str.format(). It would only really make sense > for named arguments, and I don?t think > print('{value!d}'.format(value=value) is much of a win. > > The result is a string, so if you really wanted to, you could use a > string formatting spec. So: > > print(f'*{value!d:^20}*' > > would produce: > > * value=10 * > > Although I don?t think that would be very useful in general. > > The mnemonic is !d for ?debugging?. I?d wanted to use !=, because > there?s an equal sign involved in the result, but = is the one character > that can?t be used after ! (it?s ?not equal? in expressions, and > f-strings look specifically for that case). I also mentioned !!, but I > think I prefer !d as being less confusing. > > This would be used in debugging print statements, that currently end up > looking like: > > print(f'value={value!r}') > > and would now be: > > print(f'{value!d}') > > There have been discussions about ways to specify str() vs. repr(), > using characters other than '=', adding spaces, etc. But they all end up > over-complicating what should be a simple tool, not a Swiss Army knife. > > Thoughts? > > Eric > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From eric at trueblade.com Fri May 3 11:53:02 2019 From: eric at trueblade.com (Eric V. Smith) Date: Fri, 3 May 2019 11:53:02 -0400 Subject: [Python-ideas] f-string "debug" conversion In-Reply-To: References: <3473f538-7683-28c0-c45e-3e0776bfd7ed@trueblade.com> <8b914470-a7c3-764a-7c30-a500ef1acf8a@trueblade.com> <6a0bf16d-9959-babf-bbcc-33222e570f1c@trueblade.com> <44ec8378-2a3c-e615-2a19-d02d3d10443f@trueblade.com> Message-ID: <9FA1177B-31D2-424F-AF9F-3D4CD77E41DE@trueblade.com> > On Oct 4, 2018, at 6:24 PM, Tim Peters wrote: > > [Tim] >> >> > Note that transforming >> > >> > {EXPR!d:FMT} >> > >> > into >> > >> > EXPR={repr(EXPR):FMT} >> > >> > is actually slightly more involved than transforming it into >> > >> > EXPR={EXPR:FMT} >> > >> > so I don't buy the argument that the original idea is simpler. More >> > magical and less useful, yes ;-) > > [Eric V. Smith] >> Actually, my proposal is to apply FMT to the entire result of >> EXPR={repr(EXPR)}, not just the repr(EXPR) part. I'm not sure either is >> particularly useful. I think I?ve come around to Tim?s way of thinking on this. If the format spec is basically useless (which I think it would be if it applied to either the whole string or to the repr of the value), then what?s the point of allowing it? I also want the 90% case to be addressed by just plain repr. I did consider (and implement) a version that prohibited format specs, but it seemed wrong to remove a feature that might have some clear use. So, my compromise is: - f"{expr!d}" expands to f"expr={repr(expr)}", but - f"{expr!d:spec} expands to f"expr={format(expr, spec)}" There?s some more discussion here: https://bugs.python.org/issue36774#msg341306 As Paul Moore says on that issue, this is explained as: use repr, unless you specify a format spec, then use format. > BTW, I checked, and I've never used !r, !s, or !a. So the idea that the format could apply to a string - when EXPR itself doesn't evaluate to a string - is simply foreign to me. I suppose it's natural to people who do use ![rsa] all the time - if such people exist ;-) Especially when debugging and logging, I use !r all the time. Eric -------------- next part -------------- An HTML attachment was scrubbed... URL: From ricky at teachey.org Fri May 3 12:37:56 2019 From: ricky at teachey.org (Ricky Teachey) Date: Fri, 3 May 2019 12:37:56 -0400 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses Message-ID: I just joined the ideas list today so I do not know if this has been discussed. Using dataclasses has been great for me, but a challenge is what to do when you don't *know* if the object you are using is a dataclass, and could be a variety of other classes. This could easily occur when you don't want to assume the data type being used by the user to represent their data, but wish to turn it into a dict (if an API exists to do so). Currently, you have to do something like this when you don't know if your object is a dataclass instance (I am using "dict_factory" since that is already the asdict() keyword arg): d = asdict(obj) if is_dataclass(obj) else dict_factory(obj) If it could also be a namedtuple, you might write this: d = asdict(obj) if is_dataclass(obj) else obj._asdict() if isinstance(obj, namedtuple) else dict_factory(obj) and if it could also be some other object with an .asdict() method: d = asdict(obj) if is_dataclass(obj) else obj._asdict() if isinstance(obj, namedtuple) else obj.asdict() if hasattr(obj, "asdict") else dict_factory(obj) This gets pretty silly/unwieldy somewhat quickly. The idea is: 1. identifying the various "asdict" APIs used in the standard library, and 2. include a keyword option for dataclasses.asdict() to cast a non-dataclasses object to a dict using these protocols (and falling back on factory_dict(obj) if it is provided). --- Ricky. "I've never met a Kentucky man who wasn't either thinking about going home or actually going home." - Happy Chandler -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhodri at kynesim.co.uk Fri May 3 13:00:46 2019 From: rhodri at kynesim.co.uk (Rhodri James) Date: Fri, 3 May 2019 18:00:46 +0100 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: References: Message-ID: <8499439c-346c-8647-fc0e-5eb44475e1f6@kynesim.co.uk> On 03/05/2019 17:37, Ricky Teachey wrote: > I just joined the ideas list today so I do not know if this has been > discussed. > > Using dataclasses has been great for me, but a challenge is what to do when > you don't *know* if the object you are using is a dataclass, and could be a > variety of other classes. > > This could easily occur when you don't want to assume the data type being > used by the user to represent their data, but wish to turn it into a dict > (if an API exists to do so). Conventionally you would get the caller to do this, because only the caller knows what format they have actually got. Anything else involves lots of guesswork and, as you point out, gets unweildy and silly very quickly. > The idea is: 1. identifying the various "asdict" APIs used in the standard > library, and 2. include a keyword option for dataclasses.asdict() to cast a > non-dataclasses object to a dict using these protocols (and falling back on > factory_dict(obj) if it is provided). Are "asdict" APIs all that common? I can't recall feeling much need for them, but I'm hardly a good test case :-) The problem with "casting" a generic non-dataclass object to a dict is that it's not remotely obvious how you do that. What key do you use if your object is a simple int or string? If you have even a straightforward user class aren't just using "thing.__dict__", what attributes do or do not make it into the dictionary? I'm sorry, but I'm not convinced that this is a rabbit-hole that it will be useful to dive down. -- Rhodri James *-* Kynesim Ltd From boxed at killingar.net Fri May 3 15:34:25 2019 From: boxed at killingar.net (=?utf-8?Q?Anders_Hovm=C3=B6ller?=) Date: Fri, 3 May 2019 21:34:25 +0200 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: <8499439c-346c-8647-fc0e-5eb44475e1f6@kynesim.co.uk> References: <8499439c-346c-8647-fc0e-5eb44475e1f6@kynesim.co.uk> Message-ID: <469EFBD9-D321-4D0A-80D8-E183E52F3B42@killingar.net> And what about recursive structures? I do think this is a rabbit hole worth getting into as I've written an iffy dict serializer (and differ) for an audit system at work. There are many uses for such a thing I think. But maybe we just need some informal protocol? > On 3 May 2019, at 19:00, Rhodri James wrote: > >> On 03/05/2019 17:37, Ricky Teachey wrote: >> I just joined the ideas list today so I do not know if this has been >> discussed. >> Using dataclasses has been great for me, but a challenge is what to do when >> you don't *know* if the object you are using is a dataclass, and could be a >> variety of other classes. >> This could easily occur when you don't want to assume the data type being >> used by the user to represent their data, but wish to turn it into a dict >> (if an API exists to do so). > > Conventionally you would get the caller to do this, because only the caller knows what format they have actually got. Anything else involves lots of guesswork and, as you point out, gets unweildy and silly very quickly. > >> The idea is: 1. identifying the various "asdict" APIs used in the standard >> library, and 2. include a keyword option for dataclasses.asdict() to cast a >> non-dataclasses object to a dict using these protocols (and falling back on >> factory_dict(obj) if it is provided). > > Are "asdict" APIs all that common? I can't recall feeling much need for them, but I'm hardly a good test case :-) > > The problem with "casting" a generic non-dataclass object to a dict is that it's not remotely obvious how you do that. What key do you use if your object is a simple int or string? If you have even a straightforward user class aren't just using "thing.__dict__", what attributes do or do not make it into the dictionary? > > I'm sorry, but I'm not convinced that this is a rabbit-hole that it will be useful to dive down. > > -- > Rhodri James *-* Kynesim Ltd > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From storchaka at gmail.com Sat May 4 11:51:39 2019 From: storchaka at gmail.com (Serhiy Storchaka) Date: Sat, 4 May 2019 18:51:39 +0300 Subject: [Python-ideas] Custom converters in str.format() and f-strings Message-ID: Currently str.format() and f-strings support three converters. Converting is specified by the "!" character followed by a letter which denotes the converter. s: str() r: repr() a: ascii() In some cases it would be useful to apply other converters. One obvious example is escaping special characters ("&", "<", ">", "'" and '"') for XML and HTML. Other applications may need other converters, for example escaping special characters (like \xff) for Telnet, or escaping non-BMP characters for displaying on Tk widget, or escaping delimiters and special characters for shell, or translating to other language. I do not think that it is practical to provide additional standard converters for all above cases, but what if create a special registry similar to registries for encodings and error handlers and allow a user to register custom converters? We could extend str.format() and f-strings to accept arbitrary letters after "!", or maybe even allow multi-character names of converters. It is less important for f-strings because you can use arbitrary expressions, but even in this case f"Hello, {name!x}!" or f"Hello, {name!xml}!" looks better than f"Hello, {html.escape(name)}!" or f"Hello, {x(name)}!". From steve at pearwood.info Sat May 4 14:09:59 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 5 May 2019 04:09:59 +1000 Subject: [Python-ideas] Custom converters in str.format() and f-strings In-Reply-To: References: Message-ID: <20190504180959.GG5010@ando.pearwood.info> On Sat, May 04, 2019 at 06:51:39PM +0300, Serhiy Storchaka wrote: > It is less important for f-strings because you can use arbitrary > expressions, but even in this case f"Hello, {name!x}!" or f"Hello, > {name!xml}!" looks better than f"Hello, {html.escape(name)}!" or > f"Hello, {x(name)}!". html.escape(name) is mostly self-documenting. !x is a cryptic, arbitrary code. Why !x rather than !e ("escape") or !h ("html") or !g for that matter? Compact mini-languages like regexes and format codes are compact but cryptic. They're harder to read, especially when you're reading less common codes. Probably most people could guess that %y in a date format string means "year", but how many people will recognise %G and %j without looking them up? -- Steven From ronmarti18 at gmail.com Sat May 4 14:34:27 2019 From: ronmarti18 at gmail.com (Ronie Martinez) Date: Sun, 5 May 2019 02:34:27 +0800 Subject: [Python-ideas] Proposal: Using % sign for percentage Message-ID: Good day! As Python focuses on readability, why not use % sign for actual percentages? For example: ``` rate = 0.1058 # float rate = 10.58% # percent, similar to above ``` It does not interfere with modulo operator as modulo follows a different format: ``` a = x % y ``` This looks like a small feature but it will surely set Python a level higher in terms of readability. Thanks a lot! -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric at trueblade.com Sat May 4 15:14:22 2019 From: eric at trueblade.com (Eric V. Smith) Date: Sat, 4 May 2019 15:14:22 -0400 Subject: [Python-ideas] Custom converters in str.format() and f-strings In-Reply-To: References: Message-ID: <6DF79DD4-3645-4B00-B519-4627EAB6CB7B@trueblade.com> I think PEP 501 would be a better approach to this. Eric > On May 4, 2019, at 11:51 AM, Serhiy Storchaka wrote: > > Currently str.format() and f-strings support three converters. Converting is specified by the "!" character followed by a letter which denotes the converter. > > s: str() > r: repr() > a: ascii() > > In some cases it would be useful to apply other converters. One obvious example is escaping special characters ("&", "<", ">", "'" and '"') for XML and HTML. Other applications may need other converters, for example escaping special characters (like \xff) for Telnet, or escaping non-BMP characters for displaying on Tk widget, or escaping delimiters and special characters for shell, or translating to other language. > > I do not think that it is practical to provide additional standard converters for all above cases, but what if create a special registry similar to registries for encodings and error handlers and allow a user to register custom converters? We could extend str.format() and f-strings to accept arbitrary letters after "!", or maybe even allow multi-character names of converters. > > It is less important for f-strings because you can use arbitrary expressions, but even in this case f"Hello, {name!x}!" or f"Hello, {name!xml}!" looks better than f"Hello, {html.escape(name)}!" or f"Hello, {x(name)}!". > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ From jfine2358 at gmail.com Sat May 4 15:20:16 2019 From: jfine2358 at gmail.com (Jonathan Fine) Date: Sat, 4 May 2019 20:20:16 +0100 Subject: [Python-ideas] Proposal: Using % sign for percentage In-Reply-To: References: Message-ID: Hi Ronie There's a problem with using % as you suggest. What's the meaning and thus value of >>> 300%+57 At present it's 15. You would have it equal to 60. You motivation is good, but the solution you've proposed might not work. By the way, Python does care about readability of numeric constants. It was recently (?) changed to allow >>> billion = 1_000_000_000 I do agree with you, readability counts. Thank you for your contribution. Perhaps there's a better solution. -- Jonathan -------------- next part -------------- An HTML attachment was scrubbed... URL: From ricky at teachey.org Sat May 4 15:23:54 2019 From: ricky at teachey.org (Ricky Teachey) Date: Sat, 4 May 2019 15:23:54 -0400 Subject: [Python-ideas] Proposal: Using % sign for percentage In-Reply-To: References: Message-ID: I don't have any specific knowledge about this, but it probably has not been done because it could easily cause some subtle bugs that aren't immediately obvious, and the benefits aren't that great in comparison to just using the format specification language to produce % representation. Consider, if % was a postix operator: >>> x = 1.11 >>> xmod1 = x % # whoops - forgot second argument, and no error to warn me >>> print(f"the decimal portion of x is: {xmod1}") # no error here either the modulo value is: 111.00000000000001% yuck! Another problem is: if you want a "%" sign displayed as part of the repr, this would require a new "percent" numerical subtype with it's own __repr__. Which opens up a big can of worms: - should percent instances inter-operate with floats, ints, Decimals....? - if so, what should the return value be when you add/subtract/etc them with other types? Also consider that if this was done, it would be the only postfix operator in the entire language. If you really want to do this, you could subclass float and overload the __rmod__ and just have it take an ellipses as the RHS argument, which doesn't look too awful to me: >>> xpct = 1.11 % ... # not too bad! and errors are preserved when misused... >>> f"{xpct:.0f}" '111' >>> f"{xpct:.0f}%" '111%' But IMO there's already a much better way to do all of this: >>> f"{x:.0%}" '111%' -------------- next part -------------- An HTML attachment was scrubbed... URL: From ricky at teachey.org Sat May 4 16:58:43 2019 From: ricky at teachey.org (Ricky Teachey) Date: Sat, 4 May 2019 16:58:43 -0400 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses Message-ID: > Are "asdict" APIs all that common? There are at least two in the std lib: namedtuple._asdict(), and dataclasses.asdict. I assume there are others I'm not aware of. > Conventionally you would get the caller to do this... > Anything else involves lots of guesswork... > The problem with "casting" a generic non-dataclass object to a dict is > that it's not remotely obvious how you do that. What key do you use if > your object is a simple int or string? If you have even a > straightforward user class aren't just using "thing.__dict__", what > attributes do or do not make it into the dictionary? I get the impression that what I am suggesting wasn't clear. Let me approach it from a different direction. The language already provides the mapping protocol for user defined dict-making; ie, dict(myobj). So "this is how you define dict-making" rabbit hole is not only not worth going down, it's already been addressed for a very long time. The problem I want to address is very specific and two pronged: 1. some of the std lib modules DON'T USE the mapping protocol (for various very legitimate reasons) 2. some user-defined types also don't use the mapping protocol, also often for legit reasons Therefore the proposal is not to guess anything at all. The idea is to rigorously track all the existing dict APIs that have been put into the std lib over the years into a dict-making convenience function, since, for example, dict(my_dataclass) and dict(my_namedtuple) are (legitimately) broken. One can then confidently apply that asdict() convenience function without having to resort to a complicated string of if statements that might not even catch all cases. The language already does this kind of cataloging for other things, like getting a list of all the reserved keywords. Why not catalog all the std lib dict-making APIs into a dict-making convenience function, too? For user-defined protocols, we simply leave HOW to dict-ify to the caller via the factory_dict argument (if it is provided). If it isn't provided, one of the following would happen: - raise an error -- similar or even identical to, perhaps, the error raised for dict("foo") or dict(42) - use the default dict_factory (which is dict; identical to above, but succeeds if there is a dict-casting protocol already implemented) I favor the second, but I'm not dogmatic about which of those would be better. Again: no interest in trying to guess at user defined dict-making APIs as such, aside from allowing the caller to provide that API themselves if they so wish. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat May 4 20:17:51 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 5 May 2019 10:17:51 +1000 Subject: [Python-ideas] Proposal: Using % sign for percentage In-Reply-To: References: Message-ID: <20190505001750.GH5010@ando.pearwood.info> On Sun, May 05, 2019 at 02:34:27AM +0800, Ronie Martinez wrote: > It does not interfere with modulo operator as modulo follows a different > format: > ``` > a = x % y > ``` Alas, that's not correct, because + and - are both unary operators as well as binary operators, so this: x % + y is ambiguous: x modulo (+y) (x%) plus y > This looks like a small feature but it will surely set Python a level > higher in terms of readability. It definitely does look like a very small feature, but can you demonstrate that it will be better for readability by showing some actual real code that would be improved by this? How often do you hard-code a rate into your program like this? rate = 12.75% I don't see the readability improvement from converting the decimal to a percent in my head once, when I write the code: # think: I want 12.75% so divide by 100 rate = 0.1275 versus having to convert the percent to a decimal in my head every single time I read the code: # read: rate = 12.75% # think: percent operator, not modulo, # so divide by 100, the value must be 0.1275 Percent notation is already ambiguous in real life, which is why mathematicians don't use it. This is fine: discount = sell_price * 20% but what do you expect these to do? price = cost + 20% marked_down_price = price - 15% Aside from the ambiguity with modulo, and the dubious readability improvement, I think this will trip people up when they try to calculate with it. There are two obvious meanings for the above examples: # percent unary operator has low precedence price = (cost + 20)/100 # percent unary operator has high precedence price = cost + (20/100) and neither match what people mean when they say "the price is the cost plus 20 percent". -- Steven From s at matveenko.ru Sat May 4 21:23:58 2019 From: s at matveenko.ru (Serge Matveenko) Date: Sun, 5 May 2019 04:23:58 +0300 Subject: [Python-ideas] A proper way to bring real interfaces to Python Message-ID: Hi, all! I believe, almost everybody is familiar with the `abc` package. The problem is that the ABC class is not a separate thing from `object`. So, using ABCs often results in complicated inheritance designs and even in the infamous metaclass conflict. Another problem is that ABC performs checks at the moment an object is being instantiated which isn't exactly the way one expects an interface to work. The obvious way would be to enforce the implementation at the moment `type` for that class is created, i.e. on module execution time. I'm aware there was `PEP 245 -- Python Interface Syntax` which was rejected. However, I agree with the idea that the syntax proposed in this PEP wasn't particularly "pythonish". So, I would like to propose adding a third main object called `interface` in addition to `object` and `type` and to use it to define interface objects. Such interfaces could then be used in the class definition in the following way. ``` class MyInterfaceA(interface): def foo(): pass class MyInterfaceB(interface): def bar(): pass class Implementation(implements=[MyInterfaceA, MyInterfaceB]): def foo(): pass def bar(): pass ``` As a proof of concept, I've implemented a library which implements the idea and allows to use this approach right out of the box ? https://pypi.org/project/strict-interfaces/. I would like to get any feedback for the idea and the library. Thanks to all! From rymg19 at gmail.com Sat May 4 23:40:37 2019 From: rymg19 at gmail.com (Ryan Gonzalez) Date: Sat, 4 May 2019 22:40:37 -0500 Subject: [Python-ideas] A proper way to bring real interfaces to Python In-Reply-To: References: Message-ID: Worth maybe noting that this could play interestingly with PEP 484? -- Ryan (????) Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone else https://refi64.com/ On May 4, 2019, 8:24 PM -0500, Serge Matveenko , wrote: > Hi, all! > > I believe, almost everybody is familiar with the `abc` package. > > The problem is that the ABC class is not a separate thing from > `object`. So, using ABCs often results in complicated inheritance > designs and even in the infamous metaclass conflict. > > Another problem is that ABC performs checks at the moment an object is > being instantiated which isn't exactly the way one expects an > interface to work. The obvious way would be to enforce the > implementation at the moment `type` for that class is created, i.e. on > module execution time. > > I'm aware there was `PEP 245 -- Python Interface Syntax` which was > rejected. However, I agree with the idea that the syntax proposed in > this PEP wasn't particularly "pythonish". > > So, I would like to propose adding a third main object called > `interface` in addition to `object` and `type` and to use it to define > interface objects. Such interfaces could then be used in the class > definition in the following way. > > ``` > class MyInterfaceA(interface): > def foo(): > pass > > class MyInterfaceB(interface): > def bar(): > pass > > class Implementation(implements=[MyInterfaceA, MyInterfaceB]): > def foo(): > pass > def bar(): > pass > ``` > > As a proof of concept, I've implemented a library which implements the > idea and allows to use this approach right out of the box ? > https://pypi.org/project/strict-interfaces/. > > I would like to get any feedback for the idea and the library. > > Thanks to all! > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat May 4 23:36:35 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 5 May 2019 13:36:35 +1000 Subject: [Python-ideas] A proper way to bring real interfaces to Python In-Reply-To: References: Message-ID: <20190505033634.GI5010@ando.pearwood.info> On Sun, May 05, 2019 at 04:23:58AM +0300, Serge Matveenko wrote: > Hi, all! > > I believe, almost everybody is familiar with the `abc` package. > > The problem is that the ABC class is not a separate thing from > `object`. So, using ABCs often results in complicated inheritance > designs and even in the infamous metaclass conflict. [...] > So, I would like to propose adding a third main object called > `interface` in addition to `object` and `type` and to use it to define > interface objects. Such interfaces could then be used in the class > definition in the following way. How will that solve the problem? Your `interface` object will still inherit from both object and type since everything inherits from object and all types inherit from type. > Another problem is that ABC performs checks at the moment an object is > being instantiated which isn't exactly the way one expects an > interface to work. Isn't it? That's how I expect it to work in Python, and I haven't had any problems with it so far. That's not to say there aren't problems, but you should explain what they are rather than assume that others have experienced the same issues you have. Why is the late check a problem? Are you worried about performance? It isn't that I *oppose* moving the checks to class-creation time instead of instantiation time, but I'd like to hear more about why it is a problem. > The obvious way would be to enforce the > implementation at the moment `type` for that class is created, i.e. on > module execution time. Module execution time is not necessarily when the class is created, if it matters. -- Steven From s at matveenko.ru Sat May 4 23:47:28 2019 From: s at matveenko.ru (Serge Matveenko) Date: Sun, 5 May 2019 06:47:28 +0300 Subject: [Python-ideas] A proper way to bring real interfaces to Python In-Reply-To: References: Message-ID: On Sun, May 5, 2019 at 6:40 AM Ryan Gonzalez wrote: > > Worth maybe noting that this could play interestingly with PEP 484? You're right. It already plays along with type hints in different ways. At the moment it requires an exact match of implementation method signatures with the interface including type annotations. There are some examples in test modules https://github.com/lig/python-interfaces/blob/master/tests/interfaces_test.py From s at matveenko.ru Sun May 5 00:05:54 2019 From: s at matveenko.ru (Serge Matveenko) Date: Sun, 5 May 2019 07:05:54 +0300 Subject: [Python-ideas] A proper way to bring real interfaces to Python In-Reply-To: <20190505033634.GI5010@ando.pearwood.info> References: <20190505033634.GI5010@ando.pearwood.info> Message-ID: On Sun, May 5, 2019 at 6:42 AM Steven D'Aprano wrote: > > On Sun, May 05, 2019 at 04:23:58AM +0300, Serge Matveenko wrote: > > So, I would like to propose adding a third main object called > > `interface` in addition to `object` and `type` and to use it to define > > interface objects. Such interfaces could then be used in the class > > definition in the following way. > > How will that solve the problem? Your `interface` object will still > inherit from both object and type since everything inherits from object > and all types inherit from type. In my realization, `interface` stays aside from the inheritance logic. It has its own tree of inheritance and doesn't participate in MRO of its implementations. This is the key feature and the goal. > > Another problem is that ABC performs checks at the moment an object is > > being instantiated which isn't exactly the way one expects an > > interface to work. > > Isn't it? That's how I expect it to work in Python, and I haven't had > any problems with it so far. Well, you haven't had any problems doesn't mean there are no:) > That's not to say there aren't problems, but you should explain what > they are rather than assume that others have experienced the same issues > you have. Why is the late check a problem? Are you worried about > performance? One of the problems is that ABCs doesn't require a class to implement anything. IF you have only class-methods and never instantiate the class it will be never checked. Another one is that ABCs aren't interfaces at all as they all to have actual implementations being encapsulated in them. This causes misunderstandings of the code and strange behaviors like in `collection.abc` module for instance where there is a lot of implementation and some methods are just mentioned by name. The code could be much more clear if there were only interfaces and partial implementation mixins. Overall, there is no interface implementation in Python at the moment. There is abc module indeed which is just a bunch of legacy hacks from the times when there was no possibility to do it in a better way. Now we have tools in the language to do it better and I think that it's time to rethink this now. > It isn't that I *oppose* moving the checks to class-creation time > instead of instantiation time, but I'd like to hear more about why it is > a problem. The main problem that this is just checks on an instance without any hard connection to method signatures as they defined and without any check on type annotations or ever treating a class definition as a proper type. From s at matveenko.ru Sun May 5 00:12:35 2019 From: s at matveenko.ru (Serge Matveenko) Date: Sun, 5 May 2019 07:12:35 +0300 Subject: [Python-ideas] A proper way to bring real interfaces to Python In-Reply-To: References: <20190505033634.GI5010@ando.pearwood.info> Message-ID: > Another one is that ABCs aren't interfaces at all as they all to have > actual implementations being encapsulated in them. This causes > misunderstandings of the code and strange behaviors like in > `collection.abc` module for instance where there is a lot of > implementation and some methods are just mentioned by name. The code > could be much more clear if there were only interfaces and partial > implementation mixins. I meant `importlib.abc` not `collections.abc` here. From s at matveenko.ru Sun May 5 00:17:22 2019 From: s at matveenko.ru (Serge Matveenko) Date: Sun, 5 May 2019 07:17:22 +0300 Subject: [Python-ideas] A proper way to bring real interfaces to Python In-Reply-To: <20190505033634.GI5010@ando.pearwood.info> References: <20190505033634.GI5010@ando.pearwood.info> Message-ID: On Sun, May 5, 2019 at 6:42 AM Steven D'Aprano wrote: > It isn't that I *oppose* moving the checks to class-creation time > instead of instantiation time, but I'd like to hear more about why it is > a problem. This doesn't look like moving checks from one place to another for me. It is basically how I would design interfaces in Python at the current state of the language ability to express this concept. I think it is worth mentioning the following here from the POC implementation README. ``` Design Goals * Be as strict as possible * Fail on import time * Do not mess with object and/or type inheritance * Possibility to integrate in CPython Core * Ability to use "out of the box" regardless support in an interpreter ``` From luca.baldini at pi.infn.it Sun May 5 05:58:51 2019 From: luca.baldini at pi.infn.it (Luca Baldini) Date: Sun, 5 May 2019 11:58:51 +0200 Subject: [Python-ideas] Running average and stdev in the statistics module? Message-ID: Hi here, I wonder if the idea of adding to the statistics module a class to calculate the running statistics (average and standard deviation) of a generic input data stream has ever come up in the past. The basic idea is to do the necessary book-keeping as the data are fed into the accumulator class and to be able to query the average variance of the sequence at any point in time without having to loop over the thing again. The obvious way to do that is well know, and described, e.g., in Knuth TAOCP vol 2, 3rd edition, page 232. FWIW It is something that through the years I have coded myself a myriad of times (e.g., for real-time data processing)---and maybe worth considering for addition to the standard library. For completeness, a cursory look on google brings up this fairly nice package https://pypi.org/project/runstats/ but really, the core algorithm would be trivial to code in a fashion that works with decimal and fraction objects to be integrated into the statistics module. Should this spur enough interest (and assuming that the maintainer(s) of the module are not hostile to the idea) I'd like to volunteer to put together an tentative implementation. [It's my first post on this list, so please be gentle :-)] Luca -- =============================================================================== Luca Baldini Universita' di Pisa and Istituto Nazionale di Fisica Nucleare - Sezione di Pisa Largo Bruno Pontecorvo 3, I-56127, Pisa, ITALY. phone : +39 050 2214438 fax : +39 050 2214317 e-mail : luca.baldini at pi.infn.it icq : 396247302 (Garrone) web : http://www.df.unipi.it/~baldini mirror : http://www.pi.infn.it/~lbaldini =============================================================================== From turnbull.stephen.fw at u.tsukuba.ac.jp Sun May 5 13:23:32 2019 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Mon, 6 May 2019 02:23:32 +0900 Subject: [Python-ideas] A proper way to bring real interfaces to Python In-Reply-To: References: Message-ID: <23759.7188.275920.935866@turnbull.sk.tsukuba.ac.jp> Serge Matveenko writes: > So, I would like to propose adding a third main object called > `interface` in addition to `object` and `type` and to use it to define > interface objects. Such interfaces could then be used in the class > definition in the following way. How does this compare to existing technology such as zope.interface? Why do you want it to behave differently where it does? We use that heavily (and successfully) in GNU Mailman. (yet another) Steve From zakasaih at gmail.com Sun May 5 16:08:42 2019 From: zakasaih at gmail.com (Zakariae Saih) Date: Sun, 5 May 2019 21:08:42 +0100 Subject: [Python-ideas] Adding multiple constructors on one class Message-ID: Hello python team! you should consider adding the ability to create multiple constructors on one class like c# and Java it is really useful. *Example*: i want a constructor takes 1 argument and other constructor takes 2 and another one takes none in the same class. so when i want to create an object i got a lot of options. i know i can do it by given a default value to the attribute but it will be better with multiple constructors -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Sun May 5 16:25:05 2019 From: rosuav at gmail.com (Chris Angelico) Date: Mon, 6 May 2019 06:25:05 +1000 Subject: [Python-ideas] Adding multiple constructors on one class In-Reply-To: References: Message-ID: On Mon, May 6, 2019 at 6:09 AM Zakariae Saih wrote: > > Hello python team! > you should consider adding the ability to create multiple constructors on one class like c# and Java it is really useful. > Example: > i want a constructor takes 1 argument and other constructor takes 2 and another one takes none in the same class. so when i want to create an object i got a lot of options. > i know i can do it by given a default value to the attribute but it will be better with multiple constructors > More generally: You want to be able to have multiple functions with the same name, and have something distinguish them by their signatures. That's a difficult thing to do in Python, because there are so many complexities (keyword arguments, a habit of duck typing, etc), but not impossible. What you're looking for is sometimes called "multiple dispatch", and you can find some packages on PyPI that offer this feature - usually with some restrictions, eg distinguishing functions by positional args only, or requiring an actual isinstance check rather than "thing that can be added to an integer". Have a look at some of them and see if they do what you want. ChrisA From steve at pearwood.info Sun May 5 19:54:02 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 6 May 2019 09:54:02 +1000 Subject: [Python-ideas] Adding multiple constructors on one class In-Reply-To: References: Message-ID: <20190505235349.GJ5010@ando.pearwood.info> On Sun, May 05, 2019 at 09:08:42PM +0100, Zakariae Saih wrote: > Hello python team! > you should consider adding the ability to create multiple constructors on > one class like c# and Java it is really useful. Guido's time machine strikes again -- classes can have any number of constructors, and we've had the ability to do this since at least Python 2.2 if not earlier. For example: * dict has the standard constructor dict(...) and an alternate constructor dict.fromkeys(...); * datetime, date and time objects have lots of alternate constructors, such as datetime.fromtimestamp, datetime.now, datetime.combine, datetime.strptime and more. > *Example*: > i want a constructor takes 1 argument and other constructor takes 2 and > another one takes none in the same class. so when i want to create an > object i got a lot of options. > i know i can do it by given a default value to the attribute but it will be > better with multiple constructors And indeed you are correct -- you can already do this, and sometimes it is better with multiple constructors. -- Steven From greg.ewing at canterbury.ac.nz Sun May 5 21:20:14 2019 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 06 May 2019 13:20:14 +1200 Subject: [Python-ideas] Adding multiple constructors on one class In-Reply-To: <20190505235349.GJ5010@ando.pearwood.info> References: <20190505235349.GJ5010@ando.pearwood.info> Message-ID: <5CCF8BCE.2090705@canterbury.ac.nz> Steven D'Aprano wrote: > Guido's time machine strikes again -- classes can have any number of > constructors, and we've had the ability to do this since at least Python > 2.2 if not earlier. Although I think the OP wants all the constructors to have the same name, which is more difficult to achieve in Python. Personally I'm not a fan of that kind of overloading -- I prefer different functions to have different names, so I'm happy with the status quo. -- Greg From s at matveenko.ru Mon May 6 03:22:17 2019 From: s at matveenko.ru (Serge Matveenko) Date: Mon, 6 May 2019 10:22:17 +0300 Subject: [Python-ideas] A proper way to bring real interfaces to Python In-Reply-To: <23759.7188.275920.935866@turnbull.sk.tsukuba.ac.jp> References: <23759.7188.275920.935866@turnbull.sk.tsukuba.ac.jp> Message-ID: On Sun, May 5, 2019 at 8:23 PM Stephen J. Turnbull wrote: > > Serge Matveenko writes: > > > So, I would like to propose adding a third main object called > > `interface` in addition to `object` and `type` and to use it to define > > interface objects. Such interfaces could then be used in the class > > definition in the following way. > > How does this compare to existing technology such as zope.interface? > Why do you want it to behave differently where it does? `zope.interface` has a lot of functionality that is true. However, it is just another hacky way to bring interfaces functionality to a project. My proposal is to finally bring interfaces to Python. I've provided a POC for the idea. I agree that while my POC could be usable in real life projects it lacks some functionality of `zope.interfaces`. However, bringing the basic but strict interface support to Python could greatly benefit to projects like `zope.interface` as it would allow developers to write more reusable code using core Python functionality when it suits their need and rarely depend on external libraries when they need some extra features. Also, `strict-interfaces` provides typing annotations support and could be easily be adopted in conjunction with PEP 544. From storchaka at gmail.com Mon May 6 03:49:35 2019 From: storchaka at gmail.com (Serhiy Storchaka) Date: Mon, 6 May 2019 10:49:35 +0300 Subject: [Python-ideas] More alternate constructors for builtin type Message-ID: Constructors for builtin types is too overloaded. For example, int constructor: * Converts a number (with truncation) to an integer. * Parses human readable representation of integer from string or bytes-like object. Optional base can be specified. Note that there is an alternate constructor for converting bytes to int using other way: int.frombytes(). * Without arguments returns 0. str constructor: * Converts an object to human-readable representation. * Decodes a bytes-like object using the specified encoding. * Without arguments returns an empty string. bytes constructor: * Converts a bytes-like object to a bytes object. * Creates a bytes object from an iterable if integers. * Encodes a string using the specified encoding. The same as str.encode(). * Creates a bytes object of the specified length consisting of zeros. Equals to b'\0' * n. dict constructor: * Creates a dict from a mapping. * Creates a dict from an iterable of key-value pairs. * Without arguments returns an empty dict. The problem of supporting many different types of input is that we can get wrong result instead of error, or that we can get error later, far from the place where we handle input. For example, if our function should accept arbitrary bytes-like object, and we call bytes() on the argument because we need the length and indexing, and we pass an integer instead, we will get an unexpected result. If our function expects a number, and we call int() on the argument, we may prefer to get an error if pass a string. I suggest to add limited versions of constructors as named constructors: * int.parse() -- parses string or bytes to integer. I do not know whether separate int.parsestr() and int.parsebytes() are needed. I think round(), math.trunc(), math.floor() and math.ceil() are enough for lossy converting numbers to integers. operator.index() should be used for lossless conversion. * bytes.frombuffer() -- accepts only bytes-like objects. * bytes.fromvalues() -- accepts only an iterable if integers. * dict.frommapping() -- accepts only mapping, but not key-value pairs. Uses __iter__() instead of keys() for iterating keys, and can take an optional iterable of keys. Equals to {k: m[k] for k in m} or {k: m[k] for k in keys}. * dict.fromitems() -- accepts only key-value pairs. Equals to {k: v for k, v in iterable}. From levkivskyi at gmail.com Mon May 6 10:33:38 2019 From: levkivskyi at gmail.com (Ivan Levkivskyi) Date: Mon, 6 May 2019 10:33:38 -0400 Subject: [Python-ideas] A proper way to bring real interfaces to Python In-Reply-To: References: <23759.7188.275920.935866@turnbull.sk.tsukuba.ac.jp> Message-ID: On Mon, 6 May 2019 at 03:23, Serge Matveenko wrote: > On Sun, May 5, 2019 at 8:23 PM Stephen J. Turnbull > wrote: > > > > Serge Matveenko writes: > > > > > So, I would like to propose adding a third main object called > > > `interface` in addition to `object` and `type` and to use it to define > > > interface objects. Such interfaces could then be used in the class > > > definition in the following way. > > > > How does this compare to existing technology such as zope.interface? > > Why do you want it to behave differently where it does? > > Also, `strict-interfaces` provides typing annotations support and > could be easily be adopted in conjunction with PEP 544. > I am not sure why one would need another special base class to enforce checking implementations statically. Currently mypy can already enforce implementations of Protocols/ABCs at instantiation time. I can imagine one can just add a flag (like --early-implementations) without other changes. -- Ivan -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon May 6 10:49:36 2019 From: guido at python.org (Guido van Rossum) Date: Mon, 6 May 2019 09:49:36 -0500 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: References: Message-ID: 20-25 years ago this might have been a good idea. Unfortunately there's so much code (including well-publicized example code) that I'm not sure it's a good use of anyone's time to try and fix this. Exception: I am often in need of a constructor for a bytes object from an integer using the decimal representation, e.g. bytes.fromint(42) == b"42". (Especially when migrating code from Python 2, where I've found a lot of str(n) that cannot be translated to bytes(n) but must instead be written as b"%d" % n, which is ugly and unintuitive when coming from Python 2.) On Mon, May 6, 2019 at 2:50 AM Serhiy Storchaka wrote: > Constructors for builtin types is too overloaded. > > For example, int constructor: > > * Converts a number (with truncation) to an integer. > * Parses human readable representation of integer from string or > bytes-like object. Optional base can be specified. Note that there is an > alternate constructor for converting bytes to int using other way: > int.frombytes(). > * Without arguments returns 0. > > str constructor: > > * Converts an object to human-readable representation. > * Decodes a bytes-like object using the specified encoding. > * Without arguments returns an empty string. > > bytes constructor: > > * Converts a bytes-like object to a bytes object. > * Creates a bytes object from an iterable if integers. > * Encodes a string using the specified encoding. The same as str.encode(). > * Creates a bytes object of the specified length consisting of zeros. > Equals to b'\0' * n. > > dict constructor: > > * Creates a dict from a mapping. > * Creates a dict from an iterable of key-value pairs. > * Without arguments returns an empty dict. > > The problem of supporting many different types of input is that we can > get wrong result instead of error, or that we can get error later, far > from the place where we handle input. > > For example, if our function should accept arbitrary bytes-like object, > and we call bytes() on the argument because we need the length and > indexing, and we pass an integer instead, we will get an unexpected > result. If our function expects a number, and we call int() on the > argument, we may prefer to get an error if pass a string. > > I suggest to add limited versions of constructors as named constructors: > > * int.parse() -- parses string or bytes to integer. I do not know > whether separate int.parsestr() and int.parsebytes() are needed. I think > round(), math.trunc(), math.floor() and math.ceil() are enough for lossy > converting numbers to integers. operator.index() should be used for > lossless conversion. > * bytes.frombuffer() -- accepts only bytes-like objects. > * bytes.fromvalues() -- accepts only an iterable if integers. > * dict.frommapping() -- accepts only mapping, but not key-value pairs. > Uses __iter__() instead of keys() for iterating keys, and can take an > optional iterable of keys. Equals to {k: m[k] for k in m} or {k: m[k] > for k in keys}. > * dict.fromitems() -- accepts only key-value pairs. Equals to {k: v for > k, v in iterable}. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* -------------- next part -------------- An HTML attachment was scrubbed... URL: From shadowranger+pythonideas at gmail.com Mon May 6 11:26:56 2019 From: shadowranger+pythonideas at gmail.com (Josh Rosenberg) Date: Mon, 6 May 2019 15:26:56 +0000 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: References: Message-ID: The other bytes object constructor I often find myself in need of without being able to remember how to do it is creating a a length 1 bytes object from a known ordinal. The "obvious": someordinal = ... bytes(someordinal) creates a zeroed bytes of that length, which is clearly wrong. I eventually remember that wrapping it in a tuple (or list) before passing to the bytes constructor works, but it's far from intuitive: bytes((someordinal,)) Unfortunately, the most obvious name for the alternate constructor to fill this niche is *also* bytes.fromint, which conflicts with Guido's use case. On Mon, May 6, 2019 at 2:40 PM Guido van Rossum wrote: > 20-25 years ago this might have been a good idea. Unfortunately there's so > much code (including well-publicized example code) that I'm not sure it's a > good use of anyone's time to try and fix this. > > Exception: I am often in need of a constructor for a bytes object from an > integer using the decimal representation, e.g. bytes.fromint(42) == b"42". > (Especially when migrating code from Python 2, where I've found a lot of > str(n) that cannot be translated to bytes(n) but must instead be written as > b"%d" % n, which is ugly and unintuitive when coming from Python 2.) > > On Mon, May 6, 2019 at 2:50 AM Serhiy Storchaka > wrote: > >> Constructors for builtin types is too overloaded. >> >> For example, int constructor: >> >> * Converts a number (with truncation) to an integer. >> * Parses human readable representation of integer from string or >> bytes-like object. Optional base can be specified. Note that there is an >> alternate constructor for converting bytes to int using other way: >> int.frombytes(). >> * Without arguments returns 0. >> >> str constructor: >> >> * Converts an object to human-readable representation. >> * Decodes a bytes-like object using the specified encoding. >> * Without arguments returns an empty string. >> >> bytes constructor: >> >> * Converts a bytes-like object to a bytes object. >> * Creates a bytes object from an iterable if integers. >> * Encodes a string using the specified encoding. The same as str.encode(). >> * Creates a bytes object of the specified length consisting of zeros. >> Equals to b'\0' * n. >> >> dict constructor: >> >> * Creates a dict from a mapping. >> * Creates a dict from an iterable of key-value pairs. >> * Without arguments returns an empty dict. >> >> The problem of supporting many different types of input is that we can >> get wrong result instead of error, or that we can get error later, far >> from the place where we handle input. >> >> For example, if our function should accept arbitrary bytes-like object, >> and we call bytes() on the argument because we need the length and >> indexing, and we pass an integer instead, we will get an unexpected >> result. If our function expects a number, and we call int() on the >> argument, we may prefer to get an error if pass a string. >> >> I suggest to add limited versions of constructors as named constructors: >> >> * int.parse() -- parses string or bytes to integer. I do not know >> whether separate int.parsestr() and int.parsebytes() are needed. I think >> round(), math.trunc(), math.floor() and math.ceil() are enough for lossy >> converting numbers to integers. operator.index() should be used for >> lossless conversion. >> * bytes.frombuffer() -- accepts only bytes-like objects. >> * bytes.fromvalues() -- accepts only an iterable if integers. >> * dict.frommapping() -- accepts only mapping, but not key-value pairs. >> Uses __iter__() instead of keys() for iterating keys, and can take an >> optional iterable of keys. Equals to {k: m[k] for k in m} or {k: m[k] >> for k in keys}. >> * dict.fromitems() -- accepts only key-value pairs. Equals to {k: v for >> k, v in iterable}. >> >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > -- > --Guido van Rossum (python.org/~guido) > *Pronouns: he/him/his **(why is my pronoun here?)* > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From s at matveenko.ru Mon May 6 11:30:43 2019 From: s at matveenko.ru (Serge Matveenko) Date: Mon, 6 May 2019 18:30:43 +0300 Subject: [Python-ideas] A proper way to bring real interfaces to Python In-Reply-To: References: <23759.7188.275920.935866@turnbull.sk.tsukuba.ac.jp> Message-ID: On Mon, May 6, 2019 at 5:33 PM Ivan Levkivskyi wrote: > > On Mon, 6 May 2019 at 03:23, Serge Matveenko wrote: >> >> On Sun, May 5, 2019 at 8:23 PM Stephen J. Turnbull >> wrote: >> > >> > Serge Matveenko writes: >> > >> > > So, I would like to propose adding a third main object called >> > > `interface` in addition to `object` and `type` and to use it to define >> > > interface objects. Such interfaces could then be used in the class >> > > definition in the following way. >> > >> > How does this compare to existing technology such as zope.interface? >> > Why do you want it to behave differently where it does? >> >> Also, `strict-interfaces` provides typing annotations support and >> could be easily be adopted in conjunction with PEP 544. > > > I am not sure why one would need another special base class to enforce checking implementations statically. > Currently mypy can already enforce implementations of Protocols/ABCs at instantiation time. I can imagine one can just add a flag > (like --early-implementations) without other changes. This another special base class is not for static checks. This is about checks at module execution time and generally as Steven has pointed out at class creation time. This allows to build interfaces using dynamic factories for instance and then enforce the implementation. I see how mypy, type annotations and protocols are other useful links in the same chain along with interfaces but I cannot see how mypy is the replacement for the later. From storchaka at gmail.com Mon May 6 12:13:05 2019 From: storchaka at gmail.com (Serhiy Storchaka) Date: Mon, 6 May 2019 19:13:05 +0300 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: References: Message-ID: 06.05.19 17:49, Guido van Rossum ????: > 20-25 years ago this might have been a good idea. Unfortunately there's > so much code (including well-publicized example code) that I'm not sure > it's a good use of anyone's time to try and fix this. I do not propose to change the current behavior. I propose to add new named constructors. In most cases default constructors can be used, but in cases when we use type check or different tricks to limit the type of the argument, we could use named constructors. Current named constructors: * dict.fromkeys() * int.from_bytes() * float.fromhex() * bytes.fromhex() * bytearray.fromhex() From guido at python.org Mon May 6 12:38:06 2019 From: guido at python.org (Guido van Rossum) Date: Mon, 6 May 2019 11:38:06 -0500 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: References: Message-ID: On Mon, May 6, 2019 at 11:14 AM Serhiy Storchaka wrote: > 06.05.19 17:49, Guido van Rossum ????: > > 20-25 years ago this might have been a good idea. Unfortunately there's > > so much code (including well-publicized example code) that I'm not sure > > it's a good use of anyone's time to try and fix this. > > I do not propose to change the current behavior. I propose to add new > named constructors. In most cases default constructors can be used, but > in cases when we use type check or different tricks to limit the type of > the argument, we could use named constructors. > > Current named constructors: > > * dict.fromkeys() > * int.from_bytes() > * float.fromhex() > * bytes.fromhex() > * bytearray.fromhex() > Understood. My point is that we won't be able to remove the original behavior, so we'd end up with two ways to do it. :-( -- --Guido van Rossum (python.org/~guido) *Pronouns: he/him/his **(why is my pronoun here?)* -------------- next part -------------- An HTML attachment was scrubbed... URL: From s at matveenko.ru Mon May 6 12:39:39 2019 From: s at matveenko.ru (Serge Matveenko) Date: Mon, 6 May 2019 19:39:39 +0300 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: References: Message-ID: On Mon, May 6, 2019 at 7:29 PM Guido van Rossum wrote: > > On Mon, May 6, 2019 at 11:14 AM Serhiy Storchaka wrote: >> I do not propose to change the current behavior. I propose to add new >> named constructors. In most cases default constructors can be used, but >> in cases when we use type check or different tricks to limit the type of >> the argument, we could use named constructors. >> >> Current named constructors: >> >> * dict.fromkeys() >> * int.from_bytes() >> * float.fromhex() >> * bytes.fromhex() >> * bytearray.fromhex() > > > Understood. My point is that we won't be able to remove the original behavior, so we'd end up with two ways to do it. :-( With all respect, I disagree. There are ways to evolve Python such as deprecation policies which proven to be effective. There are ways to monitor current features adoption on PyPI to see whether it is safe to remove deprecated things. I'd understand if some feature is not accepted to Python if it is kinda bad. What I refuse to accept as a user is that behavior considered bad and ready to be improved is preserved through time just because it is there already. Please, get me right. I totally agree that this will bring up two ways of performing the same thing but we can deprecate one of them, keep track of the new way adoption and finally get Python to a better state if it is really desired. From solipsis at pitrou.net Mon May 6 12:47:39 2019 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 6 May 2019 18:47:39 +0200 Subject: [Python-ideas] More alternate constructors for builtin type References: Message-ID: <20190506184739.425e8848@fsol> On Mon, 6 May 2019 19:39:39 +0300 Serge Matveenko wrote: > On Mon, May 6, 2019 at 7:29 PM Guido van Rossum wrote: > > > > On Mon, May 6, 2019 at 11:14 AM Serhiy Storchaka wrote: > >> I do not propose to change the current behavior. I propose to add new > >> named constructors. In most cases default constructors can be used, but > >> in cases when we use type check or different tricks to limit the type of > >> the argument, we could use named constructors. > >> > >> Current named constructors: > >> > >> * dict.fromkeys() > >> * int.from_bytes() > >> * float.fromhex() > >> * bytes.fromhex() > >> * bytearray.fromhex() > > > > > > Understood. My point is that we won't be able to remove the original behavior, so we'd end up with two ways to do it. :-( > > With all respect, I disagree. There are ways to evolve Python such as > deprecation policies which proven to be effective. There are ways to > monitor current features adoption on PyPI to see whether it is safe to > remove deprecated things. The main constructors for built-in types are used so pervasively that there is no hope of actually removing such deprecated behavior. Regards Antoine. From s at matveenko.ru Mon May 6 12:59:55 2019 From: s at matveenko.ru (Serge Matveenko) Date: Mon, 6 May 2019 19:59:55 +0300 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: <20190506184739.425e8848@fsol> References: <20190506184739.425e8848@fsol> Message-ID: On Mon, May 6, 2019 at 7:48 PM Antoine Pitrou wrote: > > On Mon, 6 May 2019 19:39:39 +0300 > Serge Matveenko wrote: > > With all respect, I disagree. There are ways to evolve Python such as > > deprecation policies which proven to be effective. There are ways to > > monitor current features adoption on PyPI to see whether it is safe to > > remove deprecated things. > > The main constructors for built-in types are used so pervasively that > there is no hope of actually removing such deprecated behavior. I have no intention to start a long hypothetical discussion here, really. There are a lot of things which were broken at some point even despite 2to3 crusade. Not to count: `except` syntax, restriction of `async` keyword, u-strings forth and back. Usually, It doesn't matter much why one cannot upgrade the interpreter to the next version. Often, It just stops working and forces a user to dig into dependencies mess. I agree that there is no hope in making a change when there is no intention to make this change. If this change is needed there are ways to achieve that. The path could be almost infinite but it surely cannot be walked if nobody willing to take it. From s at matveenko.ru Mon May 6 13:10:44 2019 From: s at matveenko.ru (Serge Matveenko) Date: Mon, 6 May 2019 20:10:44 +0300 Subject: [Python-ideas] Running average and stdev in the statistics module? In-Reply-To: References: Message-ID: On Sun, May 5, 2019 at 1:08 PM Luca Baldini wrote: > > Hi here, > I wonder if the idea of adding to the statistics module a class to > calculate the running statistics (average and standard deviation) of a > generic input data stream has ever come up in the past. > > The basic idea is to do the necessary book-keeping as the data are fed > into the accumulator class and to be able to query the average variance > of the sequence at any point in time without having to loop over the > thing again. The obvious way to do that is well know, and described, > e.g., in Knuth TAOCP vol 2, 3rd edition, page 232. FWIW It is something > that through the years I have coded myself a myriad of times (e.g., for > real-time data processing)---and maybe worth considering for addition to > the standard library. Personally, I would definitely use this in a number of places in the real-life code I contribute to. The problem that I have with this idea is it's not clear how to store the data in an accumulator class. What about cases with different contexts in asyncio and/or multithreading code? I would say it could be useful to allow to pass a storage implementation from a user's code to address almost any possible scenario. In that case, such an accumulator class doesn't need to be a class at all and bother with any intermediate storage. It could be a number of module-level functions providing an effective algorythm implementation for user to be able to base on. From kirillbalunov at gmail.com Mon May 6 14:51:13 2019 From: kirillbalunov at gmail.com (Kirill Balunov) Date: Mon, 6 May 2019 21:51:13 +0300 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: <20190506184739.425e8848@fsol> References: <20190506184739.425e8848@fsol> Message-ID: ??, 6 ??? 2019 ?. ? 19:48, Antoine Pitrou : > On Mon, 6 May 2019 19:39:39 +0300 > Serge Matveenko wrote: > > On Mon, May 6, 2019 at 7:29 PM Guido van Rossum > wrote: > > > > > > On Mon, May 6, 2019 at 11:14 AM Serhiy Storchaka > wrote: > > >> I do not propose to change the current behavior. I propose to add new > > >> named constructors. In most cases default constructors can be used, > but > > >> in cases when we use type check or different tricks to limit the type > of > > >> the argument, we could use named constructors. > > >> > > >> Current named constructors: > > >> > > >> * dict.fromkeys() > > >> * int.from_bytes() > > >> * float.fromhex() > > >> * bytes.fromhex() > > >> * bytearray.fromhex() > > > > > > > > > Understood. My point is that we won't be able to remove the original > behavior, so we'd end up with two ways to do it. :-( > > > > With all respect, I disagree. There are ways to evolve Python such as > > deprecation policies which proven to be effective. There are ways to > > monitor current features adoption on PyPI to see whether it is safe to > > remove deprecated things. > > The main constructors for built-in types are used so pervasively that > there is no hope of actually removing such deprecated behavior. > > Especially `bytes` constructor: `bytes(int)`, it is so convenient and obvious that in the entire Python standard library it is used only in tests of its own existence: 'cpython-master/Lib/test/test_bytes.py' on line: 1023' 'cpython-master/Lib/test/test_bytes.py' on line: 1234'. Currently, `bytes` is the most ambiguous resident in Python 3. It seems that in the final version (which is very different from the initial idea), it was created to surprise: 1. Those wishing to take it as a string in Python 2 2. Those who wish to perceive it as an array of integers from 0 to 255. 3. And those who want to see `bytes` as something else... with kind regards, -gdg -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike at selik.org Mon May 6 16:52:23 2019 From: mike at selik.org (Michael Selik) Date: Mon, 6 May 2019 13:52:23 -0700 Subject: [Python-ideas] Running average and stdev in the statistics module? In-Reply-To: References: Message-ID: I've often wanted a windowing function in itertools. One exists as a recipe in the docs. If I remember correctly, one reason this was never implemented is that the most efficient implementation changes depending on the size of the window. Use a deque(maxsize=n) for large windows and tuple slicing/concat for tiny windows. I'm not sure how the tee/zip trick compares. On Mon, May 6, 2019, 10:11 AM Serge Matveenko wrote: > On Sun, May 5, 2019 at 1:08 PM Luca Baldini > wrote: > > > > Hi here, > > I wonder if the idea of adding to the statistics module a class to > > calculate the running statistics (average and standard deviation) of a > > generic input data stream has ever come up in the past. > > > > The basic idea is to do the necessary book-keeping as the data are fed > > into the accumulator class and to be able to query the average variance > > of the sequence at any point in time without having to loop over the > > thing again. The obvious way to do that is well know, and described, > > e.g., in Knuth TAOCP vol 2, 3rd edition, page 232. FWIW It is something > > that through the years I have coded myself a myriad of times (e.g., for > > real-time data processing)---and maybe worth considering for addition to > > the standard library. > > Personally, I would definitely use this in a number of places in the > real-life code I contribute to. > > The problem that I have with this idea is it's not clear how to store > the data in an accumulator class. What about cases with different > contexts in asyncio and/or multithreading code? > I would say it could be useful to allow to pass a storage > implementation from a user's code to address almost any possible > scenario. In that case, such an accumulator class doesn't need to be a > class at all and bother with any intermediate storage. It could be a > number of module-level functions providing an effective algorythm > implementation for user to be able to base on. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Mon May 6 19:09:31 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 7 May 2019 09:09:31 +1000 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: References: Message-ID: <20190506230931.GO5010@ando.pearwood.info> On Mon, May 06, 2019 at 07:39:39PM +0300, Serge Matveenko wrote: > With all respect, I disagree. There are ways to evolve Python such as > deprecation policies which proven to be effective. There are ways to > monitor current features adoption on PyPI to see whether it is safe to > remove deprecated things. Monitoring adoption on PyPI tells you absolutely nothing about the millions of lines of Python code which is not on PyPI. Not every Python programmer writes open source software available to the public. They are not second-class citizens -- we have a responsibility to them too, and that's why we don't break backwards-compatibility lightly. > I'd understand if some feature is not accepted to Python if it is > kinda bad. What I refuse to accept as a user is that behavior > considered bad and ready to be improved is preserved through time just > because it is there already. Oh, you "refuse to accept" it do you? How nice. Compared to languages like C that have ISO standards, Python's attitude towards removing old features might be seen as recklessly fast. You should read this to get another perspective: https://www.curiousefficiency.org/posts/2011/04/musings-on-culture-of-python-dev.html > Please, get me right. I totally agree that this will bring up two ways > of performing the same thing but we can deprecate one of them, keep > track of the new way adoption and finally get Python to a better state > if it is really desired. Everything you say there is true in principle. That doesn't mean it will happen in practice. For what it's worth, I'm less concerned than Guido is about having two ways to get the same result. There's two (or three, or ...) ways to do many things in Python and "Only One Way To Do It" has never been part of the Zen of Python, it was a put-down from Perl users misrepresenting Python's philosophy. But duplication has costs: more to learn, more decisions to make, more code, more tests, more documentation. Duplicated APIs can become bloat, and bloat is not free. If a function or method doesn't bring *at least* enough benefit to outweigh the costs, then it is harmful. Code churn is not free. Forcing people to change code that works because we felt like breaking it should not be done lightly. Keeping old, suboptimal APIs versus forcing code churn is a matter of balance, and choosing the right balance is not a black and white decision to make. None of this is to say that we will never decide to deprecate or remove a feature. But it isn't clear that this proposal brings enough benefit to justify such deprecations. -- Steven From steve at pearwood.info Mon May 6 19:28:45 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 7 May 2019 09:28:45 +1000 Subject: [Python-ideas] Open parenthesis in REPL completion In-Reply-To: References: <20190425083319.GV3010@ando.pearwood.info> <23745.39334.925614.263705@turnbull.sk.tsukuba.ac.jp> <1c27e786-2d99-690f-f4b6-11486e898af8@kynesim.co.uk> Message-ID: <20190506232845.GP5010@ando.pearwood.info> On Thu, Apr 25, 2019 at 02:42:09PM +0100, Stefano Borini wrote: > PyCharm adds the parenthesis, but also adds the end parenthesis, so > the whole use of parentheses is consistent: the user has not to worry > about them. > Bash refuses to guess when it's ambiguous, and stops until you fill > the ambiguous part. In Python, functions are first-class values, so parentheses or not is *always* ambiguous. If I start typing the name of a function, there is no way for the REPR to know whether I want to call the function or refer to it as an object. Although it might *guess* that calling a function is more common than treating it as a value. > Right now, the REPL implements a mixed situation where it both assumes > your usage, and does not help you all the way through. Although we can > all agree that functions most of the time are invoked, rather than > used as is. > IMHO, either the parenthesis should not be added, or two parentheses > should be added and the cursor placed in the center (I am unsure about > the details of the REPL implementation, but I guess it's possible) at > least to have a consistent experience. How ironic if a request on Python-Ideas to *stop* adding the opening parenthesis leads to the REPR adding the closing parenthesis as well. The more I think about it, the more I suspect the least-worst solution will be to make tab-completion configurable, with three options: - no parentheses - both parentheses - only the opening parenthesis (status quo) and an API to set the behaviour at startup depending on an environment variable or a config file. Anyone want to propose a patch? -- Steven From steve at pearwood.info Mon May 6 19:38:29 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 7 May 2019 09:38:29 +1000 Subject: [Python-ideas] Open parenthesis in REPL completion In-Reply-To: <5CC23F58.3060205@canterbury.ac.nz> References: <20190425083319.GV3010@ando.pearwood.info> <5CC23F58.3060205@canterbury.ac.nz> Message-ID: <20190506233829.GQ5010@ando.pearwood.info> On Fri, Apr 26, 2019 at 11:14:32AM +1200, Greg Ewing wrote: > Steven D'Aprano wrote: > >But having the opening bracket auto-added is a small satisfaction, and > >if you're using the REPL for actual calculations and not just help(), > >the benefit probably outweighs the annoyance > > The completer could detect the help( as well and leave out > the opening paren in that case. It could, but is that special case special enough to break the rules? "Why doesn't tab completion work correctly in help? I'm trying to call help(builder()) but tab completion doesn't add the opening parenthesis to builder..." -- Steven From steve at pearwood.info Mon May 6 19:44:48 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 7 May 2019 09:44:48 +1000 Subject: [Python-ideas] A proper way to bring real interfaces to Python In-Reply-To: References: Message-ID: <20190506234448.GR5010@ando.pearwood.info> On Sun, May 05, 2019 at 04:23:58AM +0300, Serge Matveenko wrote: > So, I would like to propose adding a third main object called > `interface` in addition to `object` and `type` and to use it to define > interface objects. Having read this thread, I think the proposal is complex enough that you will need to write a PEP explaining the nature of interfaces, the problem with the status quo, and your suggested solution. I don't think a proof-of-concept class is sufficient. https://www.python.org/dev/peps/ -- Steven From cs at cskk.id.au Mon May 6 19:54:03 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Tue, 7 May 2019 09:54:03 +1000 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: <20190506184739.425e8848@fsol> References: <20190506184739.425e8848@fsol> Message-ID: <20190506235403.GA82957@cskk.homeip.net> On 06May2019 18:47, Antoine Pitrou wrote: >On Mon, 6 May 2019 19:39:39 +0300 >Serge Matveenko wrote: >> On Mon, May 6, 2019 at 7:29 PM Guido van Rossum wrote: >> > On Mon, May 6, 2019 at 11:14 AM Serhiy Storchaka wrote: >> >> I do not propose to change the current behavior. I propose to add new >> >> named constructors. In most cases default constructors can be used, but >> >> in cases when we use type check or different tricks to limit the type of >> >> the argument, we could use named constructors. >> >> >> >> Current named constructors: >> >> >> >> * dict.fromkeys() >> >> * int.from_bytes() >> >> * float.fromhex() >> >> * bytes.fromhex() >> >> * bytearray.fromhex() >> > >> > Understood. My point is that we won't be able to remove the original behavior, so we'd end up with two ways to do it. :-( >> >> With all respect, I disagree. There are ways to evolve Python such as >> deprecation policies which proven to be effective. There are ways to >> monitor current features adoption on PyPI to see whether it is safe to >> remove deprecated things. > >The main constructors for built-in types are used so pervasively that >there is no hope of actually removing such deprecated behavior. I don't find that compelling. I for one would welcome a small suite of unambiguous factories that can't be misused. bytes() can easily be misused by accident, introducing bugs and requiring debug work. I'd be very happy for my own future code to be able to take advantage of hard to misuse constructors. Of course we could all write tiny factories for these modes but (a) we'd all have to write and debug them and (b) they'de all have different spellings and signatures and (c) everyone else would not have easy access to them (yes, PyPI notwithstanding: finding a specific module in there can be haphazard) and (d) the bytes type is a natural place to have these constructors/factories. All these arguments apply to the other types too. Cheers, Cameron Simpson From oscar.j.benjamin at gmail.com Mon May 6 20:08:41 2019 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 7 May 2019 01:08:41 +0100 Subject: [Python-ideas] Running average and stdev in the statistics module? In-Reply-To: References: Message-ID: On Sun, 5 May 2019 at 11:08, Luca Baldini wrote: > > Hi here, > I wonder if the idea of adding to the statistics module a class to > calculate the running statistics (average and standard deviation) of a > generic input data stream has ever come up in the past. There was discussion around this when the PEP was written. I think this is what is alluded to in the PEP with """ There is considerable interest in including one-pass functions that can calculate multiple statistics from data in iterator form, without having to convert to a list. The experimental stats package on PyPI includes co-routine versions of statistics functions. Including these will be deferred to 3.5. """ https://www.python.org/dev/peps/pep-0450/#future-work At the time I believe there was brief discussion about what the API should look like but it was deferred for future work. For the sakes of argument how about: from statistics import RunningStats, RunningMean, RunningMax stats = RunningStats({'mean':RunningMean, 'max':RunningMax}) stats.push_many([1, 5, 2, 4]) print(stats.running_stats()) # {'mean': 3, 'max': 5} -- Oscar From steve at pearwood.info Mon May 6 20:22:59 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 7 May 2019 10:22:59 +1000 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: <20190506235403.GA82957@cskk.homeip.net> References: <20190506184739.425e8848@fsol> <20190506235403.GA82957@cskk.homeip.net> Message-ID: <20190507002259.GS5010@ando.pearwood.info> On Tue, May 07, 2019 at 09:54:03AM +1000, Cameron Simpson wrote: > On 06May2019 18:47, Antoine Pitrou wrote: [...] > >The main constructors for built-in types are used so pervasively that > >there is no hope of actually removing such deprecated behavior. > > I don't find that compelling. I for one would welcome a small suite of > unambiguous factories that can't be misused. bytes() can easily be > misused by accident, introducing bugs and requiring debug work. I'd be > very happy for my own future code to be able to take advantage of hard > to misuse constructors. There is a difference between *adding* new constructor methods, and what Antoine is saying: that we cannot realistically remove existing uses of the current constructors. I think that Antoine is right: short of another major 2to3 backwards- incompatible version, the benefit of actually removing any of the built-in constructor behaviours is too small and the cost is too great. So I think removal of existing behaviour should be off the table. Or at least, taken on a case-by-case basis. Propose a specific API you want to remove, and we'll discuss that specific API. As for adding *new* constructors: > Of course we could all write tiny factories for these modes but (a) we'd > all have to write and debug them and (b) they'de all have different > spellings and signatures Probably because everyone will want them to do something different. We've already seen two different semantics for the same desired constructor call: bytes(10) -> b'10' # like str(), but returning bytes bytes(10) -> b'\x0A' # like ord(), but returning a byte That suggests a possible pair of constructors: bytes.from_int(n) -> equivalent to b'%d' % n bytes.ord(n) -> equivalent to bytes((n,)) The proposal in this thread seems to me to be a blanket call to add new constructors everywhere, and I don't think that's appropriate. I think that each proposed new constructor should live or die on its own merits. The two above for bytes seem like simple, obvious APIs that do something useful which is otherwise a small pain point. Both are syntactic sugar for something otherwise ugly or hard to discover. I think that, if somebody is willing to do the work (it can't be me, sorry) adding two new class methods to bytes for the above two cases would be a nett win, and they should be minor enough that it doesn't need a PEP. Thoughts? -- Steven From steve at pearwood.info Mon May 6 20:39:21 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 7 May 2019 10:39:21 +1000 Subject: [Python-ideas] Running average and stdev in the statistics module? In-Reply-To: References: Message-ID: <20190507003921.GT5010@ando.pearwood.info> Hi Luca, I'm the original author of the statistics module, and I'm very interested in your idea for calculating running statistics. However feature-freeze for 3.8 is not far away (about three weeks) so I think it would have to be deferred until 3.9. But I encourage you to give some thought (either privately, or publicly here in this thread) about the features you want to see. -- Steven From shadowranger+pythonideas at gmail.com Mon May 6 20:57:49 2019 From: shadowranger+pythonideas at gmail.com (Josh Rosenberg) Date: Tue, 7 May 2019 00:57:49 +0000 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: <20190507002259.GS5010@ando.pearwood.info> References: <20190506184739.425e8848@fsol> <20190506235403.GA82957@cskk.homeip.net> <20190507002259.GS5010@ando.pearwood.info> Message-ID: bytes.ord is a bad name, given the behavior would be the opposite of ord (ord converts length one str to int, not int to length one str). PEP467 (currently deferred to 3.9 or later) does have proposals for this case, either bytes.byte (old proposal: https://legacy.python.org/dev/peps/pep-0467/#addition-of-explicit-single-byte-constructors ) or bytes.fromord/a top level built-in named bchr in the new version of the PEP ( https://www.python.org/dev/peps/pep-0467/#addition-of-bchr-function-and-explicit-single-byte-constructors ). So if that's the way we want to go, we could just push forward on PEP467. It's only a subset of Serhiy's broader proposal, though admittedly one of the cases where the existing design is unusually weak and improvements would better fill niches currently occupied by non-obvious solutions. On Tue, May 7, 2019 at 12:23 AM Steven D'Aprano wrote: > On Tue, May 07, 2019 at 09:54:03AM +1000, Cameron Simpson wrote: > > On 06May2019 18:47, Antoine Pitrou wrote: > [...] > > >The main constructors for built-in types are used so pervasively that > > >there is no hope of actually removing such deprecated behavior. > > > > I don't find that compelling. I for one would welcome a small suite of > > unambiguous factories that can't be misused. bytes() can easily be > > misused by accident, introducing bugs and requiring debug work. I'd be > > very happy for my own future code to be able to take advantage of hard > > to misuse constructors. > > There is a difference between *adding* new constructor methods, and what > Antoine is saying: that we cannot realistically remove existing uses of > the current constructors. > > I think that Antoine is right: short of another major 2to3 backwards- > incompatible version, the benefit of actually removing any of the > built-in constructor behaviours is too small and the cost is too great. > So I think removal of existing behaviour should be off the table. > > Or at least, taken on a case-by-case basis. Propose a specific API you > want to remove, and we'll discuss that specific API. > > > As for adding *new* constructors: > > > Of course we could all write tiny factories for these modes but (a) we'd > > all have to write and debug them and (b) they'de all have different > > spellings and signatures > > Probably because everyone will want them to do something different. > > We've already seen two different semantics for the same desired > constructor call: > > bytes(10) -> b'10' # like str(), but returning bytes > bytes(10) -> b'\x0A' # like ord(), but returning a byte > > That suggests a possible pair of constructors: > > bytes.from_int(n) -> equivalent to b'%d' % n > bytes.ord(n) -> equivalent to bytes((n,)) > > > The proposal in this thread seems to me to be a blanket call to add new > constructors everywhere, and I don't think that's appropriate. I think > that each proposed new constructor should live or die on its own merits. > The two above for bytes seem like simple, obvious APIs that do something > useful which is otherwise a small pain point. Both are syntactic sugar > for something otherwise ugly or hard to discover. > > I think that, if somebody is willing to do the work (it can't be me, > sorry) adding two new class methods to bytes for the above two cases > would be a nett win, and they should be minor enough that it doesn't > need a PEP. > > Thoughts? > > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pythonchb at gmail.com Tue May 7 01:41:21 2019 From: pythonchb at gmail.com (Christopher Barker) Date: Mon, 6 May 2019 22:41:21 -0700 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: References: <20190506184739.425e8848@fsol> <20190506235403.GA82957@cskk.homeip.net> <20190507002259.GS5010@ando.pearwood.info> Message-ID: Oddly, it seems everyone in this thread thinks it would be "Better" to have a bunch of constructors, ratehr than the overloading, of only we didn't have backward compatibility to worry about. I disagree -- these efficiencies are what I LIKE about python: int() tries to turn whatever you pass into into an integer -- a float, a string, whatever. Do people really think that it would be better if we all had to write: int.parse("123") Rather than just int("123") ? I know my newbie students wouldn't like that. The one exception to all this in my mind is bytes -- which has come up the most in this discussion. But that's not because the bytes constructor is overloaded but rather that the bytes object is overloaded: is is storage for binary data, which its name seems to imply? or is an old-style text type? It's probably too late to do anything about that, but if we do want to "clean up" something, that would be it. """ str(n) that cannot be translated to bytes(n) but must instead be written as b"%d" % n, """ if bytes were really well, bytes, then that would be: bytes((ord(c) for c in str(n))) or str(n).encode('ascii') ugly, I know. I kinda wish we'd simply kept a single_byte_string object (essentially a py2 string) as its own thing, separate from a bytes object, which could then keep its nominal meaning -- a sequence of individual arbitrary bytes. Oh well, lots of water under that bridge..... -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue May 7 05:17:43 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 7 May 2019 19:17:43 +1000 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: References: <20190506184739.425e8848@fsol> <20190506235403.GA82957@cskk.homeip.net> <20190507002259.GS5010@ando.pearwood.info> Message-ID: <20190507091743.GU5010@ando.pearwood.info> On Tue, May 07, 2019 at 12:57:49AM +0000, Josh Rosenberg wrote: > bytes.ord is a bad name, given the behavior would be the opposite of ord > (ord converts length one str to int, not int to length one str). D'oh! I mean, well done, that was a test and you passed! *wink* Sorry for the confusion, yes, I got ord/chr confused in my head. > PEP467 (currently deferred to 3.9 or later) does have proposals for this > case, either bytes.byte (old proposal: > https://legacy.python.org/dev/peps/pep-0467/#addition-of-explicit-single-byte-constructors > ) or bytes.fromord/a top level built-in named bchr in the new version of > the PEP ( > https://www.python.org/dev/peps/pep-0467/#addition-of-bchr-function-and-explicit-single-byte-constructors > ). So if that's the way we want to go, we could just push forward on > PEP467. My personal sense is that generating a single byte from an ordinal is not common enough to justify a new builtin, but I don't have a strong opposition to the concept as such. But I am very adverse to the suggested name. Not only is "bchr" inelegant, but now that regular strings are Unicode, we really ought to avoid perpetuating the idea that single bytes are characters and text strings are bytes more than we really have to. -- Steven From greg.ewing at canterbury.ac.nz Tue May 7 08:18:34 2019 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 08 May 2019 00:18:34 +1200 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: <20190507002259.GS5010@ando.pearwood.info> References: <20190506184739.425e8848@fsol> <20190506235403.GA82957@cskk.homeip.net> <20190507002259.GS5010@ando.pearwood.info> Message-ID: <5CD1779A.6060005@canterbury.ac.nz> Steven D'Aprano wrote: > That suggests a possible pair of constructors: > > bytes.from_int(n) -> equivalent to b'%d' % n > bytes.ord(n) -> equivalent to bytes((n,)) I don't see how bytes.from_int(n) is any clearer about what it does than just bytes(n). If we're going to have named constructors, the names should indicate *how* the construction is being done, not just repeat the type of the argument. -- Greg From steve at pearwood.info Tue May 7 08:50:49 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 7 May 2019 22:50:49 +1000 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: <5CD1779A.6060005@canterbury.ac.nz> References: <20190506184739.425e8848@fsol> <20190506235403.GA82957@cskk.homeip.net> <20190507002259.GS5010@ando.pearwood.info> <5CD1779A.6060005@canterbury.ac.nz> Message-ID: <20190507125049.GV5010@ando.pearwood.info> On Wed, May 08, 2019 at 12:18:34AM +1200, Greg Ewing wrote: > Steven D'Aprano wrote: > > >That suggests a possible pair of constructors: > > > > bytes.from_int(n) -> equivalent to b'%d' % n > > bytes.ord(n) -> equivalent to bytes((n,)) > > I don't see how bytes.from_int(n) is any clearer about what it > does than just bytes(n). If we're going to have named constructors, > the names should indicate *how* the construction is being done, > not just repeat the type of the argument. Don't be shy, give us your suggested name. I'm not married to the suggestion of "from_int", if somebody with better skills at naming than me can come up with a short but descriptive name that would be great. But my other suggestion was going to be: bytes.return_bytes_representing_the_integer_input_as_an_ascii_bytestring() *wink* But seriously, its nice when a method of function name is completely self-explanatory, and some day I hope to find one which is. But until then, names are more like mnemonics: - dict.fromkeys returns a dict created from keys in what way? - str.encode encodes a string into what kind of data? - bytes.fromhex does what precisely? - what on earth is a glob? We say "import spam", not "import module spam and bind it to the name spam". We use "import" as a short name that, once we've learned what it does, reminds us what it does. It doesn't have to document every bit of functionality, just enough to associate the name with the semantics. In this case, the problem we are trying to solve is this: - convert an int like 12345 into a byte-string like b'12345'. Imagine you're looking at the dir(bytes) to find an appropriate method that perhaps does what you want, and you see methods like: - count - decode - fromhex - isdigit - split etc, plus the suggested "fromint". Which method are you going to associate with the problem you're trying to solve? -- Steven From turnbull.stephen.fw at u.tsukuba.ac.jp Tue May 7 13:23:18 2019 From: turnbull.stephen.fw at u.tsukuba.ac.jp (Stephen J. Turnbull) Date: Wed, 8 May 2019 02:23:18 +0900 Subject: [Python-ideas] A proper way to bring real interfaces to Python In-Reply-To: References: <23759.7188.275920.935866@turnbull.sk.tsukuba.ac.jp> Message-ID: <23761.48902.382625.791916@turnbull.sk.tsukuba.ac.jp> Serge Matveenko writes: > On Sun, May 5, 2019 at 8:23 PM Stephen J. Turnbull > wrote: > > > > Serge Matveenko writes: > > > > > So, I would like to propose adding a third main object called > > > `interface` in addition to `object` and `type` and to use it to define > > > interface objects. Such interfaces could then be used in the class > > > definition in the following way. > > > > How does this compare to existing technology such as zope.interface? > > Why do you want it to behave differently where it does? > > `zope.interface` has a lot of functionality that is true. However, it > is just another hacky way to bring interfaces functionality to a > project. My proposal is to finally bring interfaces to Python. Yes, I understand that, but it isn't responsive to my question. zope.interface *does* "bring interfaces to Python", and it has a lot of functionality already. You haven't explained why zope.interfaces is "hacky", nor why it would be so beneficial to have interface support in the stdlib. > support to Python could greatly benefit to projects like > `zope.interface` as it would allow developers to write more reusable > code using core Python functionality "Could" isn't enough to support "should". Real-world examples would help. Eg, point to a stdlib module (which by definition only uses core Python functionality) that could be easier to understand and/or maintain with built-in 'interface' support. (Feel free to assume functionality that hasn't been implemented in 'strict-interface' yet, and wave your hands a lot.) The thing is, projects in one or a few files with few non-stdlib imports just don't need interface support in my experience. They're small enough to keep in my head, and the overhead of defining interfaces and implementing them separately is too great to be worthwhile. Type annotations are helpful in ensuring proper use of classes and functions. In larger projects, the overhead of importing zope.interface is proportionately tiny, and well-worth the effort to create and maintain interfaces that are heavier-weight than function signatures. From oscar.j.benjamin at gmail.com Tue May 7 14:17:00 2019 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 7 May 2019 19:17:00 +0100 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: References: <20190506184739.425e8848@fsol> <20190506235403.GA82957@cskk.homeip.net> <20190507002259.GS5010@ando.pearwood.info> Message-ID: On Tue, 7 May 2019 at 06:42, Christopher Barker wrote: > > Oddly, it seems everyone in this thread thinks it would be "Better" to have a bunch of constructors, ratehr than the overloading, of only we didn't have backward compatibility to worry about. > > I disagree -- these efficiencies are what I LIKE about python: > > int() tries to turn whatever you pass into into an integer -- a float, a string, whatever. Do people really think that it would be better if we all had to write: > > int.parse("123") > > Rather than just int("123") ? It depends whether you want to have an exception if the wrong type of input is encountered. The alternative to x=int.parse(s) might instead be if not isinstance(s, str): raise TypeError('need a string') x = int(s) One of the things I like about Python compared to e.g. C is that it often tells me when something is going wrong in some code rather than silently doing the wrong thing. The int function accepts all kinds of things e.g. >>> int('?') 2 However in my own code if that character ever got passed to int then it would definitely indicate either a bug in the code or data corruption so I'd rather have an exception. Admittedly the non-ASCII unicode digit example is not one that has actually caused me a problem but what I have had a problem with is floats. Given that a user of my code can pass in a float in place of a string the fact that int(1.5) gives 1 can lead to bugs or confusion. For precisely these reasons there is the __index__ method to get around this for the situations where you want to convert a non-int integer type to int (without accepting strings or floats). There is math.trunc for the situation where you are not parsing strings and actually do want to accept non-integer numbers and truncate them. There is no function to parse a decimal integer string without also accepting floats though. -- Oscar From rosuav at gmail.com Tue May 7 14:24:20 2019 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 8 May 2019 04:24:20 +1000 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: References: <20190506184739.425e8848@fsol> <20190506235403.GA82957@cskk.homeip.net> <20190507002259.GS5010@ando.pearwood.info> Message-ID: On Wed, May 8, 2019 at 4:17 AM Oscar Benjamin wrote: > Admittedly the non-ASCII unicode digit example is not one that has > actually caused me a problem but what I have had a problem with is > floats. Given that a user of my code can pass in a float in place of a > string the fact that int(1.5) gives 1 can lead to bugs or confusion. > ... > There is no function to parse a decimal integer string without also > accepting floats though. Explicitly parse in decimal, then. >>> int(1.5, 10) Traceback (most recent call last): File "", line 1, in TypeError: int() can't convert non-string with explicit base ChrisA From ricky at teachey.org Tue May 7 14:43:15 2019 From: ricky at teachey.org (Ricky Teachey) Date: Tue, 7 May 2019 14:43:15 -0400 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: References: Message-ID: One pain point in Python that constantly rears its head (at least for me) is this: def eat_iterable(yummy_iterable): if isinstance(yummy_iterable, str): raise TypeError("BLECH! Strings are ATOMIC in this context, mmkay??") tasty_list = list(yummy_iterable) # digest list... The design decision to make strs sequence-like was definitely a good one with a huge number of advantages. But we all know that sometimes, treating strs just like all other kinds of sequences is very inconvenient and isn't really what we want. This leads to isinstance() checking, which feels very non-python every time I do it. It would be kind of nice if list, tuple, and set had alt constructors that disallowed things that aren't REALLY sequences of individual atomic items-- like strs, and perhaps dicts as well (which is another instance check often needed to guard against errors). Perhaps: list.atomics tuple.atomics set.atomics But naming things is hard and I'm terrible at it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From oscar.j.benjamin at gmail.com Tue May 7 15:14:24 2019 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 7 May 2019 20:14:24 +0100 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: References: <20190506184739.425e8848@fsol> <20190506235403.GA82957@cskk.homeip.net> <20190507002259.GS5010@ando.pearwood.info> Message-ID: On Tue, 7 May 2019, 19:25 Chris Angelico, wrote: > On Wed, May 8, 2019 at 4:17 AM Oscar Benjamin > wrote: > > Admittedly the non-ASCII unicode digit example is not one that has > > actually caused me a problem but what I have had a problem with is > > floats. Given that a user of my code can pass in a float in place of a > > string the fact that int(1.5) gives 1 can lead to bugs or confusion. > > ... > > There is no function to parse a decimal integer string without also > > accepting floats though. > > Explicitly parse in decimal, then. > > >>> int(1.5, 10) > Traceback (most recent call last): > File "", line 1, in > TypeError: int() can't convert non-string with explicit base > Good point! Maybe that could be the solution for bytes as well: overload the constructor even more and allow the second argument to be an integer base: >>> bytes(123, 10) b'123' -- Oscar > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cs at cskk.id.au Tue May 7 18:14:50 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Wed, 8 May 2019 08:14:50 +1000 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: <5CD1779A.6060005@canterbury.ac.nz> References: <5CD1779A.6060005@canterbury.ac.nz> Message-ID: <20190507221450.GA66724@cskk.homeip.net> On 08May2019 00:18, Greg Ewing wrote: >Steven D'Aprano wrote: >>That suggests a possible pair of constructors: >> bytes.from_int(n) -> equivalent to b'%d' % n >> bytes.ord(n) -> equivalent to bytes((n,)) > >I don't see how bytes.from_int(n) is any clearer about what it >does than just bytes(n). If we're going to have named constructors, >the names should indicate *how* the construction is being done, >not just repeat the type of the argument. How about from_size(n) and from_ord(n)? The former to make a NUL filled bytes of size n, the latter to make a single byte bytes with element 0 having value n. Preemptively one could argue for from_size having an optional fill value, default 0. I think I'm -0 on that because I can't imagine a likely use case and it would slightly slow down every use. I think I'd argue for a from_iter instead, to support an arbitrary fill pattern/sequence. That avoids slowing the common case and provides flexibility at the same time. Cheers, Cameron Simpson From steve at pearwood.info Tue May 7 19:40:35 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 8 May 2019 09:40:35 +1000 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: <20190507221450.GA66724@cskk.homeip.net> References: <5CD1779A.6060005@canterbury.ac.nz> <20190507221450.GA66724@cskk.homeip.net> Message-ID: <20190507234035.GY5010@ando.pearwood.info> On Wed, May 08, 2019 at 08:14:50AM +1000, Cameron Simpson wrote: > On 08May2019 00:18, Greg Ewing wrote: > >Steven D'Aprano wrote: > >>That suggests a possible pair of constructors: > >> bytes.from_int(n) -> equivalent to b'%d' % n > >> bytes.ord(n) -> equivalent to bytes((n,)) > > > >I don't see how bytes.from_int(n) is any clearer about what it > >does than just bytes(n). If we're going to have named constructors, > >the names should indicate *how* the construction is being done, > >not just repeat the type of the argument. > > How about from_size(n) and from_ord(n)? The former to make a NUL filled > bytes of size n, the latter to make a single byte bytes with element 0 > having value n. We already have from_size, it's just spelled bytes(n). I don't dislike from_ord as a name, although perhaps it ought to be fromord to match fromhex. > Preemptively one could argue for from_size having an optional fill > value, default 0. I think I'm -0 on that because I can't imagine a > likely use case and it would slightly slow down every use. Wanting to fill a bytes object with something other than zeroes is probably uncommon. But for those who need it '\xFF'*n should do the job. > I think I'd > argue for a from_iter instead, to support an arbitrary fill > pattern/sequence. We already have that, its spelled bytes(iterable). py> bytes(range(5)) b'\x00\x01\x02\x03\x04' -- Steven From steve at pearwood.info Tue May 7 20:20:13 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 8 May 2019 10:20:13 +1000 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: References: <20190506184739.425e8848@fsol> <20190506235403.GA82957@cskk.homeip.net> <20190507002259.GS5010@ando.pearwood.info> Message-ID: <20190508002013.GZ5010@ando.pearwood.info> On Tue, May 07, 2019 at 07:17:00PM +0100, Oscar Benjamin wrote: > The int function accepts all kinds of things e.g. > > >>> int('?') > 2 > > However in my own code if that character ever got passed to int then > it would definitely indicate either a bug in the code or data > corruption so I'd rather have an exception. If you ever get Thai users who would like to enter numbers in their own language, they may be a tad annoyed that you consider that a bug. > Admittedly the non-ASCII unicode digit example is not one that has > actually caused me a problem I don't see why it should cause a problem. An int is an int, regardless of how it was spelled before conversion. You probably don't lose any sleep over the relatively high probability of a single flipped bit changing a '7' digit into a '6' digit, say. It seems strange to worry about the enormously less likely data corruption which just so happens to result in valid non-ASCII digits. If your application can support user-data of "123" for the int 123, why would it matter if the user spelled it '???' instead? You're not obligated to output Thai digits if you don't have many Thai users, but it just seems mean to reject Thai input if It Just Works. > but what I have had a problem with is > floats. Given that a user of my code can pass in a float in place of a > string the fact that int(1.5) gives 1 can lead to bugs or confusion. So write a helper function and use that. Or specify a base: int(string, 0) will support the usual Python formats (e.g. any of '123', '0x7b', '0o173', '0b1111011') without converting non-strings. If for some reason you only want to support a single base, say, base 7, you can specify a non-zero argument as the base: py> int('234', 7) 123 But user input seems like a good place to apply Postel's Law: "Be conservative in what you output, be liberal in what you accept." It shouldn't be any skin off your nose to accept '0x7b' or '???' as well as '123'. -- Steven From s at matveenko.ru Wed May 8 05:29:58 2019 From: s at matveenko.ru (Serge Matveenko) Date: Wed, 8 May 2019 12:29:58 +0300 Subject: [Python-ideas] A proper way to bring real interfaces to Python In-Reply-To: <20190506234448.GR5010@ando.pearwood.info> References: <20190506234448.GR5010@ando.pearwood.info> Message-ID: On Tue, May 7, 2019 at 2:45 AM Steven D'Aprano wrote: > > On Sun, May 05, 2019 at 04:23:58AM +0300, Serge Matveenko wrote: > > > So, I would like to propose adding a third main object called > > `interface` in addition to `object` and `type` and to use it to define > > interface objects. > > Having read this thread, I think the proposal is complex enough that > you will need to write a PEP explaining the nature of interfaces, the > problem with the status quo, and your suggested solution. I'd be happy to do that. Would you like to sponsor such a PEP? From rosuav at gmail.com Wed May 8 05:36:57 2019 From: rosuav at gmail.com (Chris Angelico) Date: Wed, 8 May 2019 19:36:57 +1000 Subject: [Python-ideas] A proper way to bring real interfaces to Python In-Reply-To: References: <20190506234448.GR5010@ando.pearwood.info> Message-ID: On Wed, May 8, 2019 at 7:30 PM Serge Matveenko wrote: > > On Tue, May 7, 2019 at 2:45 AM Steven D'Aprano wrote: > > > > On Sun, May 05, 2019 at 04:23:58AM +0300, Serge Matveenko wrote: > > > > > So, I would like to propose adding a third main object called > > > `interface` in addition to `object` and `type` and to use it to define > > > interface objects. > > > > Having read this thread, I think the proposal is complex enough that > > you will need to write a PEP explaining the nature of interfaces, the > > problem with the status quo, and your suggested solution. > > I'd be happy to do that. Would you like to sponsor such a PEP? When you come to write it up, I hope you can go into some detail about what you mean by "a third main object". Currently, the hierarchy is reentrant at the apex ("object" is an instance of "type", and "type" is a subclass of "object"); where would "interface" fit in? Would it be a type? Would it be an object? Will both type and interface be subclasses of a BaseType, same as we have with exceptions? ChrisA From s at matveenko.ru Wed May 8 05:43:07 2019 From: s at matveenko.ru (Serge Matveenko) Date: Wed, 8 May 2019 12:43:07 +0300 Subject: [Python-ideas] A proper way to bring real interfaces to Python In-Reply-To: References: <20190506234448.GR5010@ando.pearwood.info> Message-ID: On Wed, May 8, 2019 at 12:37 PM Chris Angelico wrote: > When you come to write it up, I hope you can go into some detail about > what you mean by "a third main object". Currently, the hierarchy is > reentrant at the apex ("object" is an instance of "type", and "type" > is a subclass of "object"); where would "interface" fit in? Would it > be a type? Would it be an object? Will both type and interface be > subclasses of a BaseType, same as we have with exceptions? It's a good point. Thanks for that! From cpcloud at gmail.com Wed May 8 09:48:24 2019 From: cpcloud at gmail.com (Phillip Cloud) Date: Wed, 8 May 2019 09:48:24 -0400 Subject: [Python-ideas] Specific equality method for hashable objects Message-ID: A number of libraries override the equality operator (__eq__) in such a way that it returns an object that isn't a Python bool. This all but precludes making these objects hashable, even though there's nothing in principle that prevents them from being so. The main use case I have in mind is an expression DSL that returns another expression when its __eq__ method is called (sqlalchemy is a good example of such a DSL). In such DSLs you often want to use expressions as keys in dict. You *can* use them as keys, but the code that does is only reliable insofar as it can guarantee that the expressions in the dict are identical (in the sense of a pointer) to the values you're going to later use for lookup. After thinking about this for a bit, I'd like to understand whether it's reasonable to propose a new method __hash_eq__ that would be used during dict key comparisons (in lookdict). This new method would allow an object to override __eq__ *and* be hashable, by defining __hash_eq__. __hash_eq__ would only be allowed to return a bool, unlike __eq__. Regarding backwards compatibility, __hash_eq__ would be defined by default to be equivalent to __eq__. I haven't thought about other standard library mappings like OrderedDict yet, and it's entirely possible that such considerations would kill this idea. I did some googling before writing this up, and I couldn't find anything that proposed anything like this but perhaps my google skills are lacking. If there's a history here that I'm missing I'm happy to learn more about it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Wed May 8 10:08:27 2019 From: brett at python.org (Brett Cannon) Date: Wed, 8 May 2019 10:08:27 -0400 Subject: [Python-ideas] Specific equality method for hashable objects In-Reply-To: References: Message-ID: This goes against "there's only one way to do it" and overriding equality and hashing is niche enough that I don't think it warrants having two ways of doing it (because you can't drop overriding __eq__/__hash__ due to backwards-compatibility concerns). On Wed, May 8, 2019 at 9:50 AM Phillip Cloud wrote: > A number of libraries override the equality operator (__eq__) in such a > way that it returns an object that isn't a Python bool. This all but > precludes making these objects hashable, even though there's nothing in > principle that prevents them from being so. > > The main use case I have in mind is an expression DSL that returns another > expression when its __eq__ method is called (sqlalchemy is a good example > of such a DSL). In such DSLs you often want to use expressions as keys in > dict. You *can* use them as keys, but the code that does is only reliable > insofar as it can guarantee that the expressions in the dict are identical > (in the sense of a pointer) to the values you're going to later use for > lookup. > > After thinking about this for a bit, I'd like to understand whether it's > reasonable to propose a new method __hash_eq__ that would be used during > dict key comparisons (in lookdict). > > This new method would allow an object to override __eq__ *and* be > hashable, by defining __hash_eq__. __hash_eq__ would only be allowed to > return a bool, unlike __eq__. > > Regarding backwards compatibility, __hash_eq__ would be defined by default > to be equivalent to __eq__. > > I haven't thought about other standard library mappings like OrderedDict > yet, and it's entirely possible that such considerations would kill this > idea. > > I did some googling before writing this up, and I couldn't find anything > that proposed anything like this but perhaps my google skills are lacking. > If there's a history here that I'm missing I'm happy to learn more about it. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From klahnakoski at mozilla.com Wed May 8 11:05:22 2019 From: klahnakoski at mozilla.com (Kyle Lahnakoski) Date: Wed, 08 May 2019 15:05:22 -0000 Subject: [Python-ideas] Trigonometry in degrees In-Reply-To: References: <23322.5661.552082.91600@turnbull.sk.tsukuba.ac.jp> Message-ID: <276188c8-90be-41a0-8356-b79d6a420fa2@mozilla.com> Maybe a library of trig functions that include an Angle type? This is related to the timespan units discussion we just had def sin(angle): On 2018-06-08 01:44, Yuval Greenfield wrote: > On Thu, Jun 7, 2018 at 10:38 PM Stephen J. Turnbull > > wrote: > > > 6.123233995736766e-17 > >>> > > is good enough for government work, including at the local public high > school. > > > There probably is room for a library like "fractions" that represents > multiples of pi or degrees precisely. I'm not sure how complicated or > valuable of an endeavor that would be. But while I agree that floating > point is good enough, we probably can do better. > ? > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From pythonchb at gmail.com Thu May 9 01:37:41 2019 From: pythonchb at gmail.com (Christopher Barker) Date: Wed, 8 May 2019 22:37:41 -0700 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: References: Message-ID: On Tue, May 7, 2019 at 11:43 AM Ricky Teachey wrote: > def eat_iterable(yummy_iterable): > if isinstance(yummy_iterable, str): > raise TypeError("BLECH! Strings are ATOMIC in this context, > mmkay??") > tasty_list = list(yummy_iterable) > # digest list... > > The design decision to make strs sequence-like was definitely a good one > with a huge number of advantages. But we all know that sometimes, treating > strs just like all other kinds of sequences is very inconvenient and isn't > really what we want. > I've always thought this was a wart in Python, and the one type error that does show up a lot. (with true division now, it's the only type error that shows up a lot...) But the "problem" isn't that strings are a sequence -- as you say that is handy. The problem is that there is now character type -- so a string is not a sequence of characters, it's a sequence of length-1 strings -- which leads to potentially infinite recursion as you drill down to get the single item. But introducing a character type has been rejected multiple times. Oh well. > This leads to isinstance() checking, which feels very non-python every > time I do it. > Agreed. And it's only an adequate solution because for all intents and purposes, there is only one string type -- it would be very rare to duck type a string. > It would be kind of nice if list, tuple, and set had alt constructors that > disallowed things that aren't REALLY sequences of individual atomic items > yeach! as you said, a string is a sequence, and that's a good thing. and dicts being an iterable of the keys is kind of nice, too -- and once you have that, they they should be able to be passed into into sequence constructors. (and I actually have done list(a_dict) on purpose when I needed a list of the keys in a dict. -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhodri at kynesim.co.uk Thu May 9 07:26:18 2019 From: rhodri at kynesim.co.uk (Rhodri James) Date: Thu, 9 May 2019 12:26:18 +0100 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: References: Message-ID: On 04/05/2019 21:58, Ricky Teachey wrote: >> Are "asdict" APIs all that common? > > There are at least two in the std lib: namedtuple._asdict(), and > dataclasses.asdict. I assume there are others I'm not aware of. > >> Conventionally you would get the caller to do this... >> Anything else involves lots of guesswork... > >> The problem with "casting" a generic non-dataclass object to a dict is >> that it's not remotely obvious how you do that. What key do you use if >> your object is a simple int or string? If you have even a >> straightforward user class aren't just using "thing.__dict__", what >> attributes do or do not make it into the dictionary? > > I get the impression that what I am suggesting wasn't clear. Let me > approach it from a different direction. > > The language already provides the mapping protocol for user defined > dict-making; ie, dict(myobj). So "this is how you define dict-making" > rabbit hole is not only not worth going down, it's already been > addressed for a very long time. dict(myobj) is only defined for mapping objects. For everything else, "dict-making" *isn't* addressed. (Also, it's not a protocol. I think what you want is a protocol, but you aren't specifying it well enough for me to grasp.) > The problem I want to address is very specific and two pronged: > > 1. some of the std lib modules DON'T USE the mapping protocol (for > various very legitimate reasons) > > 2. some user-defined types also don't use the mapping protocol, also > often for legit reasons > > Therefore the proposal is not to guess anything at all. The idea is to > rigorously track all the existing dict APIs that have been put into > the std lib over the years into a dict-making convenience function, > since, for example, dict(my_dataclass) and dict(my_namedtuple) are > (legitimately) broken. Actually dict() would be the place to do this, surely? If all you want is to add dataclasses and namedtuples to the list of things dict() copes with, that's entirely reasonable. If you have a list of obviously mapping-like things dict() should cope with, that's fine. If you want someone else to go do the work for you, that's less fine, but hey, someone might think it's an interesting thing to do. If you want dict() to cope with arbitrary objects (which you still appear to), you need to say how. > One can then confidently apply that asdict() convenience function > without having to resort to a complicated string of if statements that > might not even catch all cases. I still think this is an unhelpful starting place. I would actually be hesitant to use an API that just said "oh, throw anything at me, I'll cope." I'd much rather use something that told me what format it expected data in, what I need to conform to, and do any conversion myself. With half an ounce of luck I'll be able to arrange to get the data in the right format in the first place and avoid a whole unnecessary layer of processing. > For user-defined protocols, we simply leave HOW to dict-ify to the > caller via the factory_dict argument (if it is provided). I am minded of the Larson cartoon, "...and now a miracle occurs..." This is the first time you've mentioned this argument (to dict(), presumably?). > If it isn't > provided, one of the following would happen: > > - raise an error -- similar or even identical to, perhaps, the error > raised for dict("foo") or dict(42) > > - use the default dict_factory (which is dict; identical to above, > but succeeds if there is a dict-casting protocol already implemented) > > I favor the second, but I'm not dogmatic about which of those would be better. > > Again: no interest in trying to guess at user defined dict-making APIs > as such, aside from allowing the caller to provide that API themselves > if they so wish. Right. So what you want is to add a keyword argument to dict() such that dict(obj, factory_dict=dict_me_harder) calls dict_me_harder(obj) if obj is not a type dict() knows how to deal with natively, expecting it to return a new dict or raise an exception. Is that right? -- Rhodri James *-* Kynesim Ltd From ricky at teachey.org Thu May 9 09:42:54 2019 From: ricky at teachey.org (Ricky Teachey) Date: Thu, 9 May 2019 09:42:54 -0400 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: References: Message-ID: > > dict(myobj) is only defined for mapping objects. For everything else, > "dict-making" *isn't* addressed. > > (Also, it's not a protocol. I think what you want is a protocol, but > you aren't specifying it well enough for me to grasp.) Here I was talking about the "mapping protocol" (define keys() and getitem()) but as you say, this essentially turns your object INTO a mapping. That isn't the same as some dict-making API (well I guess, unless you include returning two-tuples from iter). That was bad phrasing on my part. > If all you want is to add dataclasses and namedtuples to the list of things dict() copes > with, that's entirely reasonable. If you have a list of obviously > mapping-like things dict() should cope with, that's fine. If you want > someone else to go do the work for you, that's less fine, but hey, > someone might think it's an interesting thing to do. If you want dict() > to cope with arbitrary objects (which you still appear to), you need to > say how. > At this point I am really exploring the problem and potential solutions. Your responses have been helpful. I do not have a list of obviously mapping-like things that eschew the mapping protocol, other than NT and dclses. It would be helpful to know if more of these exist. I'd be happy to try and look for them, but I don't really know how I'd do that. You bring up a good point-- dict() likely makes more sense for this than expanding the dataclasses.asdict(). Looking at your menu of choices (which I think hit on most of the potential solutions), I would change my initial approach to the problem and suggest instead that two ideas should be considered: - add NamedTuple._asdict() and dataclasses.asdict() to the list of things dict() copes with - add an alt constructor that behaves just like the dict() constructor, but "dicts harder", as you put it But I see two problems with doing it using the dict() constructor: - typing.NamedTuple and @dataclass classes can also define their own keys() and getitem()-- the behavior of which might diverge from their respective asdict() methods. So dict() would need to decide whether nt._asdict() and dataclasses.asdict() take precedence over the mapping protocol (I would suggest that IF the mapping protocol exists on an object, it would take precedence). - I am sure there is lots of code out there currently depending on errors occuring when dict() is applied to a NT. So of the two options, I think an alt constructor is far superior. The alt constructor would still need to decide the first precedence problem though. I am minded of the Larson cartoon, "...and now a miracle occurs..." > This is the first time you've mentioned this argument (to dict(), > presumably?). I would actually be hesitant to use an API that just said "oh, throw anything at me, I'll cope." Well my initial idea (now reconsidered) was to change the behavior of dataclasses.asdict(), which does already have a dict_factory kwarg. I was referring to that asdict() argument. However for me, adding a kwarg to dict() is not on the table; dict() makes use of **kwargs and reserving one for this purpose is surely a bad idea. That last comment makes sense to me, but the alternative- adding handling of NT and dclses to dict()- seems bad because of backward compatibility problem I mentioned above. Therefore because of the constructor **kwargs problem, and because I agree miracles are bad things in programming :), and because you are right that it is better for the user to present the data in an expected format, adopting a dict_factory kwarg is probably not worth bothering with. I'm to happy to abandon that. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ricky at teachey.org Thu May 9 09:58:58 2019 From: ricky at teachey.org (Ricky Teachey) Date: Thu, 9 May 2019 09:58:58 -0400 Subject: [Python-ideas] More alternate constructors for builtin type In-Reply-To: References: Message-ID: (Don't seem to be getting much interest or traction on this so I'm about ready to give up on it, but I should still respond) > yeach! as you said, a string is a sequence, and that's a good thing. > Yes but as you said: it's a sequence of infinitely recursive length 1 sequences. It's definitely a different animal. And the type checking is a chore regardless of whether it's the only string type (though there's also the byte string). and dicts being an iterable of the keys is kind of nice, too -- and once you have that, they they should be able to be passed into into sequence constructors. But the other side to this is you end up on occasion also guarding against dict with a type check, or both dict AND str-- and I could even see guarding against set. Though certainly not nearly as often as str by itself. (and I actually have done list(a_dict) on purpose when I needed a list of the keys in a dict. I think that is done all the time by lots of people. But if you are dealing with dicts, a strict alt constructor would simply be the wrong choice. Thinking about it more: if such a strict non-str-sequence constructor existed, perhaps it should also exclude sets, which aren't even sequences. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhodri at kynesim.co.uk Thu May 9 10:55:47 2019 From: rhodri at kynesim.co.uk (Rhodri James) Date: Thu, 9 May 2019 15:55:47 +0100 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: References: Message-ID: On 09/05/2019 14:42, Ricky Teachey wrote: > I do not have a list of obviously mapping-like things that eschew the > mapping protocol, other than NT and dclses. It would be helpful to know if > more of these exist. I'd be happy to try and look for them, but I don't > really know how I'd do that. The only thing I can think of is going through the standard library documentation (or the code if you prefer) and seeing what leaps out as dict-like but not actually a mapping. That sounds like a lot of work to me :-( -- Rhodri James *-* Kynesim Ltd From ricky at teachey.org Thu May 9 13:13:03 2019 From: ricky at teachey.org (Ricky Teachey) Date: Thu, 9 May 2019 13:13:03 -0400 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: References: Message-ID: Another difficulty I just thought of: - a NT class- and a dcls, for that matter- could also be an iterable two-tuples that would currently get dict-ified using iteration. So a precedence rule- does iter() win? Or the asdict functions?- would be needed for this, too. The answer to that seems a lot less obvious to me (compared to the mapping protocol), since iterating k-v pairs is relying on a bit of a side effect, but that side effect has probably existed since the dict was created. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Fri May 10 07:33:01 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 10 May 2019 21:33:01 +1000 Subject: [Python-ideas] Running average and stdev in the statistics module? In-Reply-To: References: Message-ID: <20190510113301.GC5010@ando.pearwood.info> On Mon, May 06, 2019 at 08:10:44PM +0300, Serge Matveenko wrote: > On Sun, May 5, 2019 at 1:08 PM Luca Baldini wrote: > > > > Hi here, > > I wonder if the idea of adding to the statistics module a class to > > calculate the running statistics (average and standard deviation) of a > > generic input data stream has ever come up in the past. [...] > Personally, I would definitely use this in a number of places in the > real-life code I contribute to. > > The problem that I have with this idea is it's not clear how to store > the data in an accumulator class. What about cases with different > contexts in asyncio and/or multithreading code? Can you give an example of the sort of thing you might want to do? Thanks, -- Steven From pythonchb at gmail.com Fri May 10 13:52:41 2019 From: pythonchb at gmail.com (Christopher Barker) Date: Fri, 10 May 2019 10:52:41 -0700 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: References: Message-ID: Frankly, I don't think an "as dict" protocol is worth it, but if others did think so, we could: define a new dunder: __as_mapping__ (I'm using "mapping", rather than "dict", to allow duck typing) Then the built-in dict() constructor would: check for an __as_mapping__ dunder: -- if there, it would call it and expect to get a mapping back that it could turn into a dict. -- if not, it would go through its current process ( you could perhaps specify a subset of the mapping interface -- maybe all you'd need is .items(), for instance -- so it would actually return an iterator of key-value pairs. after all, dict() can already consume that.) Meanwhile, any other code that wants to could also use the __as_mapping__ dunder to be able to work with an object as though it were a mapping, even if that isn't its usual interface. As far as I can see, the only downside of this is that it would means a new standard dunder, which is a substantial downside, as (I think) it essentially adds overhead in all sorts of places, plus ads "One more protocol" to a language already ripe with protocols. At least that's the message I got when I suggested adding a dunder for a sorting key method. -CHB On Thu, May 9, 2019 at 10:13 AM Ricky Teachey wrote: > Another difficulty I just thought of: > - a NT class- and a dcls, for that matter- could also be an iterable > two-tuples that would currently get dict-ified using iteration. So a > precedence rule- does iter() win? Or the asdict functions?- would be needed > for this, too. > > The answer to that seems a lot less obvious to me (compared to the mapping > protocol), since iterating k-v pairs is relying on a bit of a side effect, > but that side effect has probably existed since the dict was created. > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython -------------- next part -------------- An HTML attachment was scrubbed... URL: From ricky at teachey.org Fri May 10 16:22:59 2019 From: ricky at teachey.org (Ricky Teachey) Date: Fri, 10 May 2019 16:22:59 -0400 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: References: Message-ID: > > As far as I can see, the only downside of this is that it would means a > new standard dunder, which is a substantial downside, as (I think) it > essentially adds overhead in all sorts of places, plus ads "One more > protocol" to a language already ripe with protocols. > I like the __as_mapping__ idea a lot since it leaves it up to the user to determine how dict() does dict-making. But I understand how the overhead from adding a single function call could be very significant to something that is used as often as dict(). What about pairing a __mapping__ method with a mapping() function instead? mapping() would not need to be a keyword; it might make sense as part of, perhaps, the types library. types.mapping() would be significantly slower than dict() only sometimes-- and resource critical code could still rely on dict(). But it would also be more forgiving. And possibly even more useful (for the reasons I've tried to do a good job of outlining). -------------- next part -------------- An HTML attachment was scrubbed... URL: From rhodri at kynesim.co.uk Fri May 10 16:36:19 2019 From: rhodri at kynesim.co.uk (Rhodri James) Date: Fri, 10 May 2019 21:36:19 +0100 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: References: Message-ID: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> On 10/05/2019 21:22, Ricky Teachey wrote: >> >> As far as I can see, the only downside of this is that it would means a >> new standard dunder, which is a substantial downside, as (I think) it >> essentially adds overhead in all sorts of places, plus ads "One more >> protocol" to a language already ripe with protocols. >> > > I like the __as_mapping__ idea a lot since it leaves it up to the user to > determine how dict() does dict-making. But I understand how the overhead > from adding a single function call could be very significant to something > that is used as often as dict(). I come back to the point of what the use cases for this are. So far you have presented a situation in which you have made your life difficult in order to make other people's lives easier (though I'd contend it doesn't actually do that). If this is a common enough occurrence, that would justify adding the (relatively small and self-contained) functionality to dict(). I personally doubt it, but I'm ready to be proved wrong. > What about pairing a __mapping__ method with a mapping() function instead? > mapping() would not need to be a keyword; it might make sense as part of, > perhaps, the types library. > > types.mapping() would be significantly slower than dict() only sometimes-- > and resource critical code could still rely on dict(). But it would also be > more forgiving. And possibly even more useful (for the reasons I've tried > to do a good job of outlining). It would still require other types to add and maintain __mapping__ dunders. While the bar for this is lower than adding functionality to dict(), I'm still not sure you've got over it. -- Rhodri James *-* Kynesim Ltd From pythonchb at gmail.com Fri May 10 21:46:11 2019 From: pythonchb at gmail.com (Christopher Barker) Date: Fri, 10 May 2019 18:46:11 -0700 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> Message-ID: > > > It would still require other types to add and maintain __mapping__ > dunders. Sure ? but that is inherent in this whole idea ? nothing other than the type object itself could possibly know how to dictify itself. Though yeah, that?s the key question? how often is this useful??? -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython -------------- next part -------------- An HTML attachment was scrubbed... URL: From ricky at teachey.org Fri May 10 22:27:55 2019 From: ricky at teachey.org (Ricky Teachey) Date: Fri, 10 May 2019 22:27:55 -0400 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> Message-ID: > > Sure ? but that is inherent in this whole idea ? nothing other than the > type object itself could possibly know how to dictify itself. > This seems to have an obvious solution: object.__as_mapping__() could simply call dict(self). Anything that uses the mapping protocol could then be mappified. Though yeah, that?s the key question? how often is this useful??? > I'll try to make a quick case it is. Dataclasses have existed for barely a year. I don't know how widely used they are yet, by I sure use the beschtick out of them. The API is very nice and I fully expect them to be wildly successful. But the dclses API encourages the use of the asdict function rather than the mapping protocol-- if they become successful and widely used, it will become more and more inconvenient to have to test for multiple mapping APIs. And this isn't the first alt mapping API. NT is already incredibly successful (to the degree it has been an intention of the core devs to use it more and more in core python libraries), and as such there is a ton of if isinstance(obj,NT) obj._asdict() else dict(obj) code out there. Another dictish type in the std lib is types.SimpleNamespace, which is typically dictified (or rather, re-dictified) via yet another API, dict(obj.__dict__). The more we pile these APIs upon APIs, the more ridiculous it gets to create a flexible API of your own that allows the user to choose what kind of structure representation THEY want to provide. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ja.py at farowl.co.uk Sat May 11 06:31:19 2019 From: ja.py at farowl.co.uk (Jeff Allen) Date: Sat, 11 May 2019 11:31:19 +0100 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> Message-ID: On 11/05/2019 02:46, Christopher Barker wrote: > > Though yeah, that?s the key question? how often is this useful??? > I have wanted something similar to this in Jython. Java supplies a battery of containers (with different performance and concurrency characteristics) and we monkepatch onto them the methods of list, set or dict according to their interface as java.util.List, Set or Map. This works "slightly badly", because sometimes those Python types have methods with the same name as one in the underlying container, but a different meaning. E.g. list.pop() disgrees about which end. I'd much rather have some kind of explicit wrapping or cast into their Python type: I had imagined the wrapped object would identify and behave as exactly that Python type (or ABC), however, it would be a *proxy* to the original. Thus one obtains a "view" onto the original collection, or looked at another way, a way to choose the implementation behind a particular Python dict, set or list. Several alternative ways of expressing this come to mind, but .asdict() is one. It's not always clear in this discussion whether a proxy is envisaged (where a put() goes through to the original object, if allowed) or just a way of constructing and filling a new dict. The latter would not fit my imagined use. Jeff Jeff Allen -------------- next part -------------- An HTML attachment was scrubbed... URL: From haael at interia.pl Sat May 11 13:20:15 2019 From: haael at interia.pl (haael) Date: Sat, 11 May 2019 19:20:15 +0200 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> Message-ID: <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Python allows for breaking out from a loop through 'break' and 'continue' keywords. It would be nice if it was possible to break many loop levels using one command. I propose two constructions for that: break break break break break break ... continue break continue break break continue ... And so on. Example: for i in range(10): for j in range(10): if i == 2 and j == 3: break break for i in range(10): for j in range(10): if i == 2 and j == 3: break continue if i == 7: break break Breaking out from many loops at once is a common practice, currently implemented through boolean testing and exceptions. This proposal would make it cleaner. From jfine2358 at gmail.com Sat May 11 13:33:24 2019 From: jfine2358 at gmail.com (Jonathan Fine) Date: Sat, 11 May 2019 18:33:24 +0100 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Message-ID: Dear Haael With python you can write for i, j in something (): # loop body This would cover both your examples. For the first, use itertools.product. For the second, write a custom iterator. -- Jonathan On Sat, 11 May 2019, 18:22 haael, wrote: > > Python allows for breaking out from a loop through 'break' and > 'continue' keywords. It would be nice if it was possible to break many > loop levels using one command. > > I propose two constructions for that: > > break > break break > break break break > ... > > continue > break continue > break break continue > ... > > And so on. > > Example: > > for i in range(10): > for j in range(10): > if i == 2 and j == 3: > break break > > for i in range(10): > for j in range(10): > if i == 2 and j == 3: > break continue > if i == 7: > break break > > Breaking out from many loops at once is a common practice, currently > implemented through boolean testing and exceptions. This proposal would > make it cleaner. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Sat May 11 13:38:57 2019 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 11 May 2019 18:38:57 +0100 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Message-ID: On Sat, 11 May 2019 at 18:22, haael wrote: > Breaking out from many loops at once is a common practice Do you have evidence for this? In my experience, it's very rare (although I concede it would be *occasionally* convenient). In most cases where I'd consider breaking out of multiple levels of nested loop, I've tended to find that it's probably more readable to refactor the code to use a generator, or something like that. Paul From pythonchb at gmail.com Sat May 11 15:50:42 2019 From: pythonchb at gmail.com (Christopher Barker) Date: Sat, 11 May 2019 12:50:42 -0700 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> Message-ID: On Sat, May 11, 2019 at 4:06 AM Jeff > I'd much rather have some kind of explicit wrapping or cast into their Python type: I had imagined the wrapped object would identify and behave as exactly that Python type (or ABC), however, it would be a *proxy* to the original. The ABCs are already defined ? if you want to make a wrapper proxy object, you implement the ABC you want to emulate. Now that I?m thinking about it more, this really covers most use cases: if you want your object to be converted to a dict by passing it to dict(), then implement the iteration protocol, returning key-value pairs that you want. This would work fine for dataclasses, and SimpleNamespaces, if one wanted to implement that. The only problem here is if you want to use the standard iteration protocol for something different -- I can't image what that would be. Sure, on the general case, maybe, but for a class that has a "natural" use case for dictifying, how else would you want to iterate over it?? -CHB > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brenbarn at brenbarn.net Sat May 11 15:59:48 2019 From: brenbarn at brenbarn.net (Brendan Barnwell) Date: Sat, 11 May 2019 12:59:48 -0700 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> Message-ID: <5CD729B4.2020703@brenbarn.net> On 2019-05-11 12:50, Christopher Barker wrote: > On Sat, May 11, 2019 at 4:06 AM Jeff > > > I'd much rather have some kind of explicit wrapping or cast into > their Python type: I had imagined the wrapped object would identify and > behave as exactly that Python type (or ABC), however, it would be a > *proxy* to the original. > > The ABCs are already defined ? if you want to make a wrapper proxy > object, you implement the ABC you want to emulate. > > > Now that I?m thinking about it more, this really covers most use cases: > if you want your object to be converted to a dict by passing it to > dict(), then implement the iteration protocol, returning key-value pairs > that you want. This would work fine for dataclasses, and > SimpleNamespaces, if one wanted to implement that. > > The only problem here is if you want to use the standard iteration > protocol for something different -- I can't image what that would be. > Sure, on the general case, maybe, but for a class that has a "natural" > use case for dictifying, how else would you want to iterate over it?? One example would be. . . dicts. Calling dict on a dict gives you a dict, but iterating over the dict just gives you the keys. Certain other mapping types (such as Pandas Series and DataFrames) also have this behavior where iterating over the mapping gives you only the keys. -- Brendan Barnwell "Do not follow where the path may lead. Go, instead, where there is no path, and leave a trail." --author unknown From pythonchb at gmail.com Sat May 11 16:51:47 2019 From: pythonchb at gmail.com (Christopher Barker) Date: Sat, 11 May 2019 13:51:47 -0700 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: <5CD729B4.2020703@brenbarn.net> References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <5CD729B4.2020703@brenbarn.net> Message-ID: On Sat, May 11, 2019 at 1:38 PM Brendan Barnwell wrote: > > protocol for something different -- I can't image what that would be. > > Sure, on the general case, maybe, but for a class that has a "natural" > > use case for dictifying, how else would you want to iterate over it?? > > One example would be. . . dicts. Calling dict on a dict gives you > a > dict, but iterating over the dict just gives you the keys. Certain > other mapping types (such as Pandas Series and DataFrames) also have > this behavior where iterating over the mapping gives you only the keys. > well, there is that ... :-) And that IS the mapping protocol. I guess i was thinking that you had an object that was NOT a Mapping already. If it IS a Mapping, then presumably it already can be passed into the dict constructor, yes? So if you want your class "dictable", then you can implement either the Mapping ABC, or the iteration protocol. I"m having an even harder time imagining why you would want a Mapping interface that isn't the same as what you'd want turned into a dict. So why again do we yet another way to do it? Anyway, I was playing with this, and write a prototype decorator that makes a datacalss "dictable" -- that it, it can be passed in to the dict constructor to make a dict. This is done by providing an iterator that returns (key, value) pairs. The code is in this git rep, if anyone wants to comment on it or improve, it, or... https://github.com/PythonCHB/PythonTopics/blob/master/Sources/code/dict_making/dictable_dataclass.py (also here for your reading pleasure) I'm not sure how the dict constructor decides an input is a mapping or an iterable -- maybe I'll play with that now. -CHB #!/usr/bin/env python """ A prototype of a decorator that adds an iterator to a dataclass so it can be passed in to the dict() constructor to make a dict. If this is thought to be useful it could be added to the dataclass decorator itself, to give all decorators this functionality. """ from dataclasses import dataclass class DataClassIterator: """ Iterator for dataclasses This used the class' __dataclass_fields__ dict to iterate through the fields and their values """ def __init__(self, dataclass_instance): self.dataclass_instance = dataclass_instance self.keys_iter = iter(dataclass_instance.__dataclass_fields__.keys()) def __iter__(self): return self def __next__(self): key = next(self.keys_iter) return (key, getattr(self.dataclass_instance, key)) def _dunder_iter(self): """ function used as the __iter__ method in the dictable_dataclass decorator """ return DataClassIterator(self) def dictable_dataclass(the_dataclass): """ class decorator for making a dataclass iterable in a way that is compatible with the dict() constructor """ the_dataclass.__iter__ = _dunder_iter return the_dataclass # Example from the dataclass docs: @dictable_dataclass @dataclass class InventoryItem: '''Class for keeping track of an item in inventory.''' name: str unit_price: float quantity_on_hand: int = 0 def total_cost(self) -> float: return self.unit_price * self.quantity_on_hand if __name__ == "__main__": # try it out: inv_item = InventoryItem("sneakers", 50.0, 20) print("an InventoryItem:\n", inv_item) print() print("And the dict you can make from it:") print(dict(inv_item)) > -- > Brendan Barnwell > "Do not follow where the path may lead. Go, instead, where there is no > path, and leave a trail." > --author unknown > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython -------------- next part -------------- An HTML attachment was scrubbed... URL: From mertz at gnosis.cx Sat May 11 17:20:23 2019 From: mertz at gnosis.cx (David Mertz) Date: Sat, 11 May 2019 17:20:23 -0400 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Message-ID: I don't love the 'break break' syntax idea. But I find myself wanting to break out of nested loops quite often. Very rarely could this be transformed to an 'itertools.permutation' de-nesting, albeit perhaps more often than I actually do that. My usual kludge is a sentinel 'break_outer' variable that gets set back and forth between True and False working the loops. Usually work a name that descriptive of the actual domain of the code. On Sat, May 11, 2019, 1:40 PM Paul Moore wrote: > On Sat, 11 May 2019 at 18:22, haael wrote: > > Breaking out from many loops at once is a common practice > > Do you have evidence for this? In my experience, it's very rare > (although I concede it would be *occasionally* convenient). > > In most cases where I'd consider breaking out of multiple levels of > nested loop, I've tended to find that it's probably more readable to > refactor the code to use a generator, or something like that. > Paul > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ricocotam at gmail.com Sat May 11 17:25:54 2019 From: ricocotam at gmail.com (Adrien Ricocotam) Date: Sat, 11 May 2019 23:25:54 +0200 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Message-ID: <4975CEEF-94BC-49B3-B131-0AD2EBA90D9B@gmail.com> > On 11 May 2019, at 23:20, David Mertz wrote: > > I don't love the 'break break' syntax idea. But I find myself wanting to break out of nested loops quite often. Very rarely could this be transformed to an 'itertools.permutation' de-nesting, albeit perhaps more often than I actually do that. > Do you have an example of such loops ? From pythonchb at gmail.com Sat May 11 18:14:24 2019 From: pythonchb at gmail.com (Christopher Barker) Date: Sat, 11 May 2019 15:14:24 -0700 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <5CD729B4.2020703@brenbarn.net> Message-ID: OK, so some more exploration: Other may already know this, but it seems that the dict constructor essentially checks if the passed in object is a Mapping ABC: if it is, it iterates on teh keys and used __getitem__ to get the values. if it is not, it iterates over key, value pairs. So if you want your custom class to present the Mapping interface, and be able to be passed in to dict(), then you shoudl subclass from collections.Mapping.ABC If you do not want your class to present the Mapping interface, then your __Iter__ should return an iterator that returns key.value pairs. Here is some code that explores that with a custom Mapping class. https://github.com/PythonCHB/PythonTopics/blob/master/Sources/code/dict_making/dict_making.py (also below) The trick here is that with, e.g. the dataclasses decorator, I'm not sure how to make it derive from the Mapping ABC -- I'm sure it's doable, but I haven't dug into that yet. Personally, I'm a big fan of duck typing -- so rather than check for the Mapping ABC, I think it should check for .items() first, and use that if it's there, and then use the regular iterator otherwise. Then all you'd have to do to make your class "dictable" would be to provide an items() iterator. But again -- there are two ways to make your classes dictable -- why do we need a third? -CHB Sample code: """ an exploration of how the dict constructor knows whether it is working with a MApping of a general iterable. It looks like the code is something like this: if isinstance(input, collections.abc.Mapping): self.update = {key: input[key] key in input} else: self.update = {key: val for key, val in input} So if your custom class present themapping interface -- that is, iterating over it returns the keys, than you want to be a MApping ABC subclass.ABC But if your custom class is not a Mapping, then you want __iter__ to return an iterator over teh key, value pairs. """ from collections.abc import Mapping def test_iter(instance): yield ("this") yield ("that") class DictTest(Mapping): def __init__(self, this="this", that="that"): self.this = this self.that = that def __iter__(self): print("iter called") return test_iter(self) def __getitem__(self, key): return getattr(self, key) def __len__(self): print("__len__ called") if __name__ == "__main__": dt = DictTest() print(dict(dt)) dt = DictTest(this=45, that=654) print(dict(dt)) On Sat, May 11, 2019 at 1:51 PM Christopher Barker wrote: > On Sat, May 11, 2019 at 1:38 PM Brendan Barnwell > wrote: > >> > protocol for something different -- I can't image what that would be. >> > Sure, on the general case, maybe, but for a class that has a "natural" >> > use case for dictifying, how else would you want to iterate over it?? >> >> One example would be. . . dicts. Calling dict on a dict gives >> you a >> dict, but iterating over the dict just gives you the keys. Certain >> other mapping types (such as Pandas Series and DataFrames) also have >> this behavior where iterating over the mapping gives you only the keys. >> > > well, there is that ... :-) And that IS the mapping protocol. I guess i > was thinking that you had an object that was NOT a Mapping already. > > If it IS a Mapping, then presumably it already can be passed into the dict > constructor, yes? > > So if you want your class "dictable", then you can implement either the > Mapping ABC, or the iteration protocol. I"m having an even harder time > imagining why you would want a Mapping interface that isn't the same as > what you'd want turned into a dict. > > So why again do we yet another way to do it? > > Anyway, I was playing with this, and write a prototype decorator that > makes a datacalss "dictable" -- that it, it can be passed in to the dict > constructor to make a dict. This is done by providing an iterator that > returns (key, value) pairs. > > The code is in this git rep, if anyone wants to comment on it or improve, > it, or... > > > https://github.com/PythonCHB/PythonTopics/blob/master/Sources/code/dict_making/dictable_dataclass.py > (also here for your reading pleasure) > > I'm not sure how the dict constructor decides an input is a mapping or an > iterable -- maybe I'll play with that now. > > -CHB > > #!/usr/bin/env python > > """ > A prototype of a decorator that adds an iterator to a dataclass > so it can be passed in to the dict() constructor to make a dict. > > If this is thought to be useful it could be added to the dataclass > decorator itself, to give all decorators this functionality. > """ > > > from dataclasses import dataclass > > > class DataClassIterator: > """ > Iterator for dataclasses > > This used the class' __dataclass_fields__ dict to iterate through the > fields and their values > """ > > def __init__(self, dataclass_instance): > self.dataclass_instance = dataclass_instance > self.keys_iter = > iter(dataclass_instance.__dataclass_fields__.keys()) > > def __iter__(self): > return self > > def __next__(self): > key = next(self.keys_iter) > return (key, getattr(self.dataclass_instance, key)) > > > def _dunder_iter(self): > """ > function used as the __iter__ method in the dictable_dataclass > decorator > """ > return DataClassIterator(self) > > > def dictable_dataclass(the_dataclass): > """ > class decorator for making a dataclass iterable in a way that is > compatible > with the dict() constructor > """ > the_dataclass.__iter__ = _dunder_iter > > return the_dataclass > > > # Example from the dataclass docs: > @dictable_dataclass > @dataclass > class InventoryItem: > '''Class for keeping track of an item in inventory.''' > name: str > unit_price: float > quantity_on_hand: int = 0 > > def total_cost(self) -> float: > return self.unit_price * self.quantity_on_hand > > > if __name__ == "__main__": > # try it out: > inv_item = InventoryItem("sneakers", 50.0, 20) > > print("an InventoryItem:\n", inv_item) > print() > > print("And the dict you can make from it:") > print(dict(inv_item)) > > > > > > > > > > > >> -- >> Brendan Barnwell >> "Do not follow where the path may lead. Go, instead, where there is no >> path, and leave a trail." >> --author unknown >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > -- > Christopher Barker, PhD > > Python Language Consulting > - Teaching > - Scientific Software Development > - Desktop GUI and Web Development > - wxPython, numpy, scipy, Cython > -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython -------------- next part -------------- An HTML attachment was scrubbed... URL: From pythonchb at gmail.com Sat May 11 18:15:24 2019 From: pythonchb at gmail.com (Christopher Barker) Date: Sat, 11 May 2019 15:15:24 -0700 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: <4975CEEF-94BC-49B3-B131-0AD2EBA90D9B@gmail.com> References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> <4975CEEF-94BC-49B3-B131-0AD2EBA90D9B@gmail.com> Message-ID: judicious use of the for loop "else" may help here, too -- I know I often forget it's there. -CHB On Sat, May 11, 2019 at 2:26 PM Adrien Ricocotam wrote: > > > > On 11 May 2019, at 23:20, David Mertz wrote: > > > > I don't love the 'break break' syntax idea. But I find myself wanting to > break out of nested loops quite often. Very rarely could this be > transformed to an 'itertools.permutation' de-nesting, albeit perhaps more > often than I actually do that. > > > > Do you have an example of such loops ? > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython -------------- next part -------------- An HTML attachment was scrubbed... URL: From mr.eightnoteight at gmail.com Sat May 11 18:22:08 2019 From: mr.eightnoteight at gmail.com (srinivas devaki) Date: Sun, 12 May 2019 03:52:08 +0530 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> <4975CEEF-94BC-49B3-B131-0AD2EBA90D9B@gmail.com> Message-ID: On Sun, May 12, 2019 at 3:45 AM Christopher Barker wrote: > > judicious use of the for loop "else" may help here, too -- I know I often forget it's there. yeah, I often do this when I want to break twice but this method only supports breaking twice at once but this doesn't support break once and break twice semantics simultaneously. ``` for x in range(n): for y in range(m): break else: continue break ``` On Sun, May 12, 2019 at 3:45 AM Christopher Barker wrote: > judicious use of the for loop "else" may help here, too -- I know I often > forget it's there. > > -CHB > > > On Sat, May 11, 2019 at 2:26 PM Adrien Ricocotam > wrote: > >> >> >> > On 11 May 2019, at 23:20, David Mertz wrote: >> > >> > I don't love the 'break break' syntax idea. But I find myself wanting >> to break out of nested loops quite often. Very rarely could this be >> transformed to an 'itertools.permutation' de-nesting, albeit perhaps more >> often than I actually do that. >> > >> >> Do you have an example of such loops ? >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > > > -- > Christopher Barker, PhD > > Python Language Consulting > - Teaching > - Scientific Software Development > - Desktop GUI and Web Development > - wxPython, numpy, scipy, Cython > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -- Regards Srinivas Devaki Software Developer at Zomato, New Delhi Phone: +91 9491 383 249 Telegram: @eightnoteight -------------- next part -------------- An HTML attachment was scrubbed... URL: From eric at trueblade.com Sat May 11 18:25:54 2019 From: eric at trueblade.com (Eric V. Smith) Date: Sat, 11 May 2019 18:25:54 -0400 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <5CD729B4.2020703@brenbarn.net> Message-ID: <8168695C-971F-4EA3-B45B-F0147EAAC592@trueblade.com> > On May 11, 2019, at 4:51 PM, Christopher Barker wrote: > >> On Sat, May 11, 2019 at 1:38 PM Brendan Barnwell wrote: > >> > protocol for something different -- I can't image what that would be. >> > Sure, on the general case, maybe, but for a class that has a "natural" >> > use case for dictifying, how else would you want to iterate over it?? >> >> One example would be. . . dicts. Calling dict on a dict gives you a >> dict, but iterating over the dict just gives you the keys. Certain >> other mapping types (such as Pandas Series and DataFrames) also have >> this behavior where iterating over the mapping gives you only the keys. > > well, there is that ... :-) And that IS the mapping protocol. I guess i was thinking that you had an object that was NOT a Mapping already. > > If it IS a Mapping, then presumably it already can be passed into the dict constructor, yes? > > So if you want your class "dictable", then you can implement either the Mapping ABC, or the iteration protocol. I"m having an even harder time imagining why you would want a Mapping interface that isn't the same as what you'd want turned into a dict. > > So why again do we yet another way to do it? > > Anyway, I was playing with this, and write a prototype decorator that makes a datacalss "dictable" -- that it, it can be passed in to the dict constructor to make a dict. This is done by providing an iterator that returns (key, value) pairs. > > The code is in this git rep, if anyone wants to comment on it or improve, it, or... > > https://github.com/PythonCHB/PythonTopics/blob/master/Sources/code/dict_making/dictable_dataclass.py > (also here for your reading pleasure) > > I'm not sure how the dict constructor decides an input is a mapping or an iterable -- maybe I'll play with that now. It?s a design goal of dataclasses to not be iterable. https://www.python.org/dev/peps/pep-0557/#why-not-just-use-namedtuple Eric > > -CHB > > #!/usr/bin/env python > > """ > A prototype of a decorator that adds an iterator to a dataclass > so it can be passed in to the dict() constructor to make a dict. > > If this is thought to be useful it could be added to the dataclass > decorator itself, to give all decorators this functionality. > """ > > > from dataclasses import dataclass > > > class DataClassIterator: > """ > Iterator for dataclasses > > This used the class' __dataclass_fields__ dict to iterate through the > fields and their values > """ > > def __init__(self, dataclass_instance): > self.dataclass_instance = dataclass_instance > self.keys_iter = iter(dataclass_instance.__dataclass_fields__.keys()) > > def __iter__(self): > return self > > def __next__(self): > key = next(self.keys_iter) > return (key, getattr(self.dataclass_instance, key)) > > > def _dunder_iter(self): > """ > function used as the __iter__ method in the dictable_dataclass decorator > """ > return DataClassIterator(self) > > > def dictable_dataclass(the_dataclass): > """ > class decorator for making a dataclass iterable in a way that is compatible > with the dict() constructor > """ > the_dataclass.__iter__ = _dunder_iter > > return the_dataclass > > > # Example from the dataclass docs: > @dictable_dataclass > @dataclass > class InventoryItem: > '''Class for keeping track of an item in inventory.''' > name: str > unit_price: float > quantity_on_hand: int = 0 > > def total_cost(self) -> float: > return self.unit_price * self.quantity_on_hand > > > if __name__ == "__main__": > # try it out: > inv_item = InventoryItem("sneakers", 50.0, 20) > > print("an InventoryItem:\n", inv_item) > print() > > print("And the dict you can make from it:") > print(dict(inv_item)) > > > > > > > > > > >> -- >> Brendan Barnwell >> "Do not follow where the path may lead. Go, instead, where there is no >> path, and leave a trail." >> --author unknown >> _______________________________________________ >> Python-ideas mailing list >> Python-ideas at python.org >> https://mail.python.org/mailman/listinfo/python-ideas >> Code of Conduct: http://python.org/psf/codeofconduct/ > > > -- > Christopher Barker, PhD > > Python Language Consulting > - Teaching > - Scientific Software Development > - Desktop GUI and Web Development > - wxPython, numpy, scipy, Cython > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcgoble3 at gmail.com Sat May 11 18:34:24 2019 From: jcgoble3 at gmail.com (Jonathan Goble) Date: Sat, 11 May 2019 18:34:24 -0400 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Message-ID: On Sat, May 11, 2019 at 1:22 PM haael wrote: > Python allows for breaking out from a loop through 'break' and > 'continue' keywords. It would be nice if it was possible to break many > loop levels using one command. I'm surprised no one has yet mentioned splitting off the loops into a function and using "return" to break out of those loops. If you nest the function in the spot where the loops would be and use nonlocal statements, you can even modify variables in the surrounding code (counters, etc.) or leave values behind. If you're opposed to a function and performance is not critical [1], you can also raise a custom "exception" at the multi-break spot and handle it at the appropriate place in the outer code. This is useful if you need to break out of a function plus additional nesting layers. [1] I mention performance here because if I recall correctly, exception handling is quite expensive and slow in Python. From tjreedy at udel.edu Sat May 11 18:57:07 2019 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 11 May 2019 18:57:07 -0400 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Message-ID: On 5/11/2019 1:20 PM, haael wrote: > > Python allows for breaking out from a loop through 'break' and > 'continue' keywords. It would be nice if it was possible to break many > loop levels using one command. > > I propose two constructions for that: > > break > break break > break break break > ... > > continue > break continue > break break continue > ... > > And so on. > > Example: > > for i in range(10): > ??? for j in range(10): > ??????? if i == 2 and j == 3: > ??????????? break break > > for i in range(10): > ??? for j in range(10): > ??????? if i == 2 and j == 3: > ??????????? break continue > ??????? if i == 7: > ??????????? break break > Breaking out from many loops at once is a common practice, currently > implemented through boolean testing and exceptions. Python exceptions generalize both 'return' and 'break' as methods for escaping nested contexts and are intended for generalized flow control. > This proposal would make it cleaner. I actually think that the general method is 'cleaner', be separating 'leave here' and 'arrive here'. This proposal would mean typing fewer words in the special case where there are nested loops within one function. The cost is a) more syntax to learn and b) tightly tying together the two loops more tightly than might be necessary, and thereby inhibiting re-factoring. In the examples above, where the loops are inherently tied together, the two loops can, as noted by others, be reduced to one. If they are not inherently tied together, one might want to reduce the inner loop to a comprehension or wrap it in a new function so it can also be used elsewhere. If the inner loop raises, wrapping or unwrapping it, as needed for reusability or efficiency, does not affect the logic. -- Terry Jan Reedy From ricky at teachey.org Sat May 11 20:13:09 2019 From: ricky at teachey.org (Ricky Teachey) Date: Sat, 11 May 2019 20:13:09 -0400 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: <8168695C-971F-4EA3-B45B-F0147EAAC592@trueblade.com> References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <5CD729B4.2020703@brenbarn.net> <8168695C-971F-4EA3-B45B-F0147EAAC592@trueblade.com> Message-ID: Using iter() for the sole purpose of being able to shoehorn your user class into being dict-cast-able is, IMO, an abuse of iter. If your class makes logical sense as an iterable of key/value pairs, then for petesake, just use the mapping protocol (implement getitem and keys). iter() is for constructing sequences, not mappings. And if the class DOESN'T make logical sense as an interable of key/value pairs, it still isn't time to abuse iter() for the convenience of making dicts. Just make an asdict() method. The above, I believe, is great API advice that I didn't come up with myself-- and it's so great that it has been followed by both NT and dclses. But the downside is, we now have the inconvenience of multiple dict-making APIs. -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Sat May 11 20:22:55 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 12 May 2019 10:22:55 +1000 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <5CD729B4.2020703@brenbarn.net> <8168695C-971F-4EA3-B45B-F0147EAAC592@trueblade.com> Message-ID: On Sun, May 12, 2019 at 10:14 AM Ricky Teachey wrote: > > Using iter() for the sole purpose of being able to shoehorn your user class into being dict-cast-able is, IMO, an abuse of iter. > > If your class makes logical sense as an iterable of key/value pairs, then for petesake, just use the mapping protocol (implement getitem and keys). iter() is for constructing sequences, not mappings. > iter() is for collections of all sorts; a sequence is specifically an ordered and indexed collection, but you can iterate over many things, including sets - and mappings. There's nothing wrong with iterable mappings, and in fact, I would be surprised (and not in a good way) by any that wasn't. ChrisA From mertz at gnosis.cx Sat May 11 22:42:33 2019 From: mertz at gnosis.cx (David Mertz) Date: Sat, 11 May 2019 22:42:33 -0400 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Message-ID: Terry reminds me of a common case I encounter that cannot be transformed into a loop over itertools.product(). E.g. for main in stuff: if thing_about(main): for detail in more_stuff(stuff, main): if seen_enough(detail): # break to outer somehow else: for detail in different_stuff(): if seen_enough(detail): # break to outer somehow On Sat, May 11, 2019, 6:58 PM Terry Reedy wrote: > On 5/11/2019 1:20 PM, haael wrote: > > > > Python allows for breaking out from a loop through 'break' and > > 'continue' keywords. It would be nice if it was possible to break many > > loop levels using one command. > > > > I propose two constructions for that: > > > > break > > break break > > break break break > > ... > > > > continue > > break continue > > break break continue > > ... > > > > And so on. > > > > Example: > > > > for i in range(10): > > for j in range(10): > > if i == 2 and j == 3: > > break break > > > > for i in range(10): > > for j in range(10): > > if i == 2 and j == 3: > > break continue > > if i == 7: > > break break > > > Breaking out from many loops at once is a common practice, currently > > implemented through boolean testing and exceptions. > > Python exceptions generalize both 'return' and 'break' as methods for > escaping nested contexts and are intended for generalized flow control. > > > This proposal would make it cleaner. > > I actually think that the general method is 'cleaner', be separating > 'leave here' and 'arrive here'. This proposal would mean typing fewer > words in the special case where there are nested loops within one function. > > The cost is a) more syntax to learn and b) tightly tying together the > two loops more tightly than might be necessary, and thereby inhibiting > re-factoring. In the examples above, where the loops are inherently > tied together, the two loops can, as noted by others, be reduced to one. > If they are not inherently tied together, one might want to reduce the > inner loop to a comprehension or wrap it in a new function so it can > also be used elsewhere. If the inner loop raises, wrapping or > unwrapping it, as needed for reusability or efficiency, does not affect > the logic. > > -- > Terry Jan Reedy > > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Sat May 11 22:50:07 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 12 May 2019 12:50:07 +1000 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Message-ID: On Sun, May 12, 2019 at 12:43 PM David Mertz wrote: > > Terry reminds me of a common case I encounter that cannot be transformed into a loop over itertools.product(). E.g. > > for main in stuff: > if thing_about(main): > for detail in more_stuff(stuff, main): > if seen_enough(detail): > # break to outer somehow > else: > for detail in different_stuff(): > if seen_enough(detail): > # break to outer somehow > For that kind of loop, there's a different problem, which is the duplication. So I would start by rewriting that as: for main in stuff: if thing_about(main): details = more_stuff(stuff, main) else: details = different_stuff() for detail in details: if seen_enough(detail): ... I'm not sure what you mean by "break to outer somehow", though, so I don't know what you're actually needing here. Concrete examples would help. ChrisA From ronmarti18 at gmail.com Sat May 11 23:01:06 2019 From: ronmarti18 at gmail.com (Ronie Martinez) Date: Sun, 12 May 2019 11:01:06 +0800 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Message-ID: I remembered, there is a "labeled break" in Perl. https://perldoc.perl.org/functions/last.html On Sun, May 12, 2019 at 10:50 AM Chris Angelico wrote: > On Sun, May 12, 2019 at 12:43 PM David Mertz wrote: > > > > Terry reminds me of a common case I encounter that cannot be transformed > into a loop over itertools.product(). E.g. > > > > for main in stuff: > > if thing_about(main): > > for detail in more_stuff(stuff, main): > > if seen_enough(detail): > > # break to outer somehow > > else: > > for detail in different_stuff(): > > if seen_enough(detail): > > # break to outer somehow > > > > For that kind of loop, there's a different problem, which is the > duplication. So I would start by rewriting that as: > > for main in stuff: > if thing_about(main): > details = more_stuff(stuff, main) > else: > details = different_stuff() > for detail in details: > if seen_enough(detail): > ... > > I'm not sure what you mean by "break to outer somehow", though, so I > don't know what you're actually needing here. Concrete examples would > help. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pythonchb at gmail.com Sat May 11 23:12:59 2019 From: pythonchb at gmail.com (Christopher Barker) Date: Sat, 11 May 2019 20:12:59 -0700 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: <8168695C-971F-4EA3-B45B-F0147EAAC592@trueblade.com> References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <5CD729B4.2020703@brenbarn.net> <8168695C-971F-4EA3-B45B-F0147EAAC592@trueblade.com> Message-ID: On Sat, May 11, 2019 at 3:26 PM Eric V. Smith wrote: > It?s a design goal of dataclasses to not be iterable. > > https://www.python.org/dev/peps/pep-0557/#why-not-just-use-namedtuple > you would know, but that reference talks about why they are not the same as NamedTuple. if dataclasses *were*iterable, they almost certainly wouldn't iterate over the values alone. And NamesTuple was designed to BE tuples -- that is a drop in replacement for tuples -- so they have the features they have partially for that reason. And my toy code actually adds another decorator to make dataclasses iterable, so it would be a completely optional feature. All that being said, I'm not actually advocating doing this with dataclasses -- I'm simply making the point that there are currretly two ways to make custom class able to be passed in to the dict constructor -- do we need another? -CHB -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython -------------- next part -------------- An HTML attachment was scrubbed... URL: From mertz at gnosis.cx Sat May 11 23:15:48 2019 From: mertz at gnosis.cx (David Mertz) Date: Sat, 11 May 2019 23:15:48 -0400 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Message-ID: Ok, sure. But what if different seen_enough() conditions are in the two inner loops? By "break to outer" ... I mean, don't get any more 'main' from 'stuff'. I meant to dig through some real code, but forgot because I was writing code for work. This example on my tablet is the general pattern I often need though. And obviously I've solved it one way or another hundreds of times. I think a "labelled break" would be nice though. It might have even been a past PEP. On Sat, May 11, 2019, 10:51 PM Chris Angelico wrote: > On Sun, May 12, 2019 at 12:43 PM David Mertz wrote: > > > > Terry reminds me of a common case I encounter that cannot be transformed > into a loop over itertools.product(). E.g. > > > > for main in stuff: > > if thing_about(main): > > for detail in more_stuff(stuff, main): > > if seen_enough(detail): > > # break to outer somehow > > else: > > for detail in different_stuff(): > > if seen_enough(detail): > > # break to outer somehow > > > > For that kind of loop, there's a different problem, which is the > duplication. So I would start by rewriting that as: > > for main in stuff: > if thing_about(main): > details = more_stuff(stuff, main) > else: > details = different_stuff() > for detail in details: > if seen_enough(detail): > ... > > I'm not sure what you mean by "break to outer somehow", though, so I > don't know what you're actually needing here. Concrete examples would > help. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Sat May 11 23:21:13 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 12 May 2019 13:21:13 +1000 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Message-ID: On Sun, May 12, 2019 at 1:16 PM David Mertz wrote: > > Ok, sure. But what if different seen_enough() conditions are in the two inner loops? By "break to outer" ... I mean, don't get any more 'main' from 'stuff'. I meant to dig through some real code, but forgot because I was writing code for work. This example on my tablet is the general pattern I often need though. And obviously I've solved it one way or another hundreds of times. > > I think a "labelled break" would be nice though. It might have even been a past PEP. > This is why concrete examples are much easier to discuss, heh. In any case, it's often possible to either (a) redefine the inner loop as some sort of container operation, or (b) redefine the outer operation as a function, and use "return". So it would be helpful to have an example that CAN'T be rewritten in one of those ways, to use as a compelling example. ChrisA From eric at trueblade.com Sat May 11 23:23:02 2019 From: eric at trueblade.com (Eric V. Smith) Date: Sat, 11 May 2019 23:23:02 -0400 Subject: [Python-ideas] add an additional dataclasses.asdict option for non-dataclasses In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <5CD729B4.2020703@brenbarn.net> <8168695C-971F-4EA3-B45B-F0147EAAC592@trueblade.com> Message-ID: > On May 11, 2019, at 11:12 PM, Christopher Barker wrote: > >> On Sat, May 11, 2019 at 3:26 PM Eric V. Smith wrote: > >> It?s a design goal of dataclasses to not be iterable. >> >> https://www.python.org/dev/peps/pep-0557/#why-not-just-use-namedtuple > > you would know, but that reference talks about why they are not the same as NamedTuple. That section mentions why they?re not iterable. Search on ?iterable?. > > if dataclasses *were*iterable, they almost certainly wouldn't iterate over the values alone. That wouldn?t make a difference. The given NT example would still be a problem. > > And NamesTuple was designed to BE tuples -- that is a drop in replacement for tuples -- so they have the features they have partially for that reason. > > And my toy code actually adds another decorator to make dataclasses iterable, so it would be a completely optional feature. That approach of adding support for iterability makes sense to me. I?m contemplating adding a ?dataclasses_tools? package for some other tools I have. But I?m not sure how this fits in to the asdict discussion. Eric > > All that being said, I'm not actually advocating doing this with dataclasses -- I'm simply making the point that there are currretly two ways to make custom class able to be passed in to the dict constructor -- do we need another? > > -CHB > > > -- > Christopher Barker, PhD > > Python Language Consulting > - Teaching > - Scientific Software Development > - Desktop GUI and Web Development > - wxPython, numpy, scipy, Cython > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mertz at gnosis.cx Sat May 11 23:29:59 2019 From: mertz at gnosis.cx (David Mertz) Date: Sat, 11 May 2019 23:29:59 -0400 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Message-ID: I'll try to find a concrete one tomorrow. I did this recently, but I might have factored it away somehow. The real life code has lots of extraneous parts that need to be removed or simplified though. Both because they aren't FLOSS, and because the details would muddy things. "Trimmed from real" won't look much different than "invented from scratch". And yes, OF COURSE everything CAN be rewritten. But that's true in regards to *every* proposed addition. The sentinel break_outer variable is often the was to do it without really big restructuring (but with a handful of ugly bookkeeping lines instead). On Sat, May 11, 2019, 11:22 PM Chris Angelico wrote: > On Sun, May 12, 2019 at 1:16 PM David Mertz wrote: > > > > Ok, sure. But what if different seen_enough() conditions are in the two > inner loops? By "break to outer" ... I mean, don't get any more 'main' from > 'stuff'. I meant to dig through some real code, but forgot because I was > writing code for work. This example on my tablet is the general pattern I > often need though. And obviously I've solved it one way or another hundreds > of times. > > > > I think a "labelled break" would be nice though. It might have even been > a past PEP. > > > > This is why concrete examples are much easier to discuss, heh. > > In any case, it's often possible to either (a) redefine the inner loop > as some sort of container operation, or (b) redefine the outer > operation as a function, and use "return". So it would be helpful to > have an example that CAN'T be rewritten in one of those ways, to use > as a compelling example. > > ChrisA > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rosuav at gmail.com Sat May 11 23:36:23 2019 From: rosuav at gmail.com (Chris Angelico) Date: Sun, 12 May 2019 13:36:23 +1000 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Message-ID: On Sun, May 12, 2019 at 1:30 PM David Mertz wrote: > > I'll try to find a concrete one tomorrow. I did this recently, but I might have factored it away somehow. The real life code has lots of extraneous parts that need to be removed or simplified though. Both because they aren't FLOSS, and because the details would muddy things. "Trimmed from real" won't look much different than "invented from scratch". > > And yes, OF COURSE everything CAN be rewritten. But that's true in regards to *every* proposed addition. The sentinel break_outer variable is often the was to do it without really big restructuring (but with a handful of ugly bookkeeping lines instead). > Yes - that's exactly the sort of thing that makes a great example. Show the way you'd LIKE to write it, and then show how you have to write it using current tools. If the latter is really ugly, requires finicky bookkeeping, or is error-prone, it's a good argument in favour of the new feature. ChrisA From eltrhn at gmail.com Sun May 12 04:36:28 2019 From: eltrhn at gmail.com (Elias Tarhini) Date: Sun, 12 May 2019 01:36:28 -0700 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Message-ID: If I may propose `break n` as a replacement for the original message's `break break ... break`, where n>0 is the number of contiguous loops to break out of and `break 1` is synonymous with `break`. Seems easier on my eyes/noggin than counting out the individual `break` statements. Eli On Sat, May 11, 2019, 10:21 AM haael wrote: > > Python allows for breaking out from a loop through 'break' and > 'continue' keywords. It would be nice if it was possible to break many > loop levels using one command. > > I propose two constructions for that: > > break > break break > break break break > ... > > continue > break continue > break break continue > ... > > And so on. > > Example: > > for i in range(10): > for j in range(10): > if i == 2 and j == 3: > break break > > for i in range(10): > for j in range(10): > if i == 2 and j == 3: > break continue > if i == 7: > break break > > Breaking out from many loops at once is a common practice, currently > implemented through boolean testing and exceptions. This proposal would > make it cleaner. > > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From phd at phdru.name Sun May 12 05:16:21 2019 From: phd at phdru.name (Oleg Broytman) Date: Sun, 12 May 2019 11:16:21 +0200 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Message-ID: <20190512091621.wyenlqovwx7kwu3m@phdru.name> On Sun, May 12, 2019 at 01:36:28AM -0700, Elias Tarhini wrote: > If I may propose `break n` as a replacement for the original message's > `break break ... break`, where n>0 is the number of contiguous loops to > break out of and `break 1` is synonymous with `break`. Seems easier on my > eyes/noggin than counting out the individual `break` statements. This is very much error-prone because on any refactoring (increasing or decreasing the number of loop levels) one must increase/decrease all numbers in internal loops. Labels are at least stable. > Eli Oleg. -- Oleg Broytman https://phdru.name/ phd at phdru.name Programmers don't die, they just GOSUB without RETURN. From steve at pearwood.info Sun May 12 05:44:44 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 12 May 2019 19:44:44 +1000 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: <20190512091621.wyenlqovwx7kwu3m@phdru.name> References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> <20190512091621.wyenlqovwx7kwu3m@phdru.name> Message-ID: <20190512094444.GE5010@ando.pearwood.info> On Sun, May 12, 2019 at 11:16:21AM +0200, Oleg Broytman wrote: > On Sun, May 12, 2019 at 01:36:28AM -0700, Elias Tarhini wrote: > > If I may propose `break n` as a replacement for the original message's > > `break break ... break`, where n>0 is the number of contiguous loops to > > break out of and `break 1` is synonymous with `break`. Seems easier on my > > eyes/noggin than counting out the individual `break` statements. > > This is very much error-prone because on any refactoring (increasing > or decreasing the number of loop levels) one must increase/decrease all > numbers in internal loops. > > Labels are at least stable. Indeed. Go has labels for loops: Label: for { ... break Label } as does Rust: 'label: while condition: { ... break label } We can't use the same syntax in Python, but we might write it like this: @label while condition: block break label and similar with for loops: @outer for x in range(100): @inner for y in range(3): if x == y == 1: break # like ``break inner`` if x == y == 2: break outer print(x, y) # break inner jumps to here # break outer jumps to here -- Steven From ronmarti18 at gmail.com Sun May 12 05:55:12 2019 From: ronmarti18 at gmail.com (Ronie Martinez) Date: Sun, 12 May 2019 17:55:12 +0800 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: <20190512094444.GE5010@ando.pearwood.info> References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> <20190512091621.wyenlqovwx7kwu3m@phdru.name> <20190512094444.GE5010@ando.pearwood.info> Message-ID: What if labels are placed in comments beside the loops, like how type is, at least, being used by analyzers? for x in range(100): # label: outer for y in range(3): # label: inner On Sun, May 12, 2019 at 5:45 PM Steven D'Aprano wrote: > On Sun, May 12, 2019 at 11:16:21AM +0200, Oleg Broytman wrote: > > On Sun, May 12, 2019 at 01:36:28AM -0700, Elias Tarhini < > eltrhn at gmail.com> wrote: > > > If I may propose `break n` as a replacement for the original message's > > > `break break ... break`, where n>0 is the number of contiguous loops to > > > break out of and `break 1` is synonymous with `break`. Seems easier on > my > > > eyes/noggin than counting out the individual `break` statements. > > > > This is very much error-prone because on any refactoring (increasing > > or decreasing the number of loop levels) one must increase/decrease all > > numbers in internal loops. > > > > Labels are at least stable. > > Indeed. > > Go has labels for loops: > > Label: > for { ... > break Label > } > > as does Rust: > > 'label: while condition: > { > ... > break label > } > > We can't use the same syntax in Python, but we might write it like this: > > @label while condition: > block > break label > > and similar with for loops: > > @outer for x in range(100): > @inner for y in range(3): > if x == y == 1: break # like ``break inner`` > if x == y == 2: break outer > print(x, y) > # break inner jumps to here > # break outer jumps to here > > > -- > Steven > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > -------------- next part -------------- An HTML attachment was scrubbed... URL: From haael at interia.pl Sun May 12 09:20:24 2019 From: haael at interia.pl (haael) Date: Sun, 12 May 2019 15:20:24 +0200 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> Message-ID: The concrete example I was working on when I started to miss double break. This is an implementation of polynomial long division in Galois field. Almost unmodified. With outer break, I would't need to use the `running` variable. In fact, for mathematical clarity, I would like to put a test in the outer loop, like: `while dividend`. It would make clear what this algorithm is actually doing. Sure, it could be refactored, but it would lose the relative simplicity. For such operations it is sometimes important to remain close to the raw mathematical notation. def __divmod__(self, other): dividend = self.canonical() try: divisor = other.canonical() except AttributeError: return NotImplemented if not dividend: return Field.zero(self.size), Field.zero(self.size) try: d = next(iter(divisor)) # leading term of the divisor except TypeError: d = self.const(divisor) do = self.__monomial_order(d) result = Field.zero(self.size) running = True while running: # outer loop, retry the changing `dividend` for x in dividend: # yield all monomials from this polynomial if self.__monomial_order(x) < do: # encountered monomial of too small order, finish running = False break # exit outer loop try: c = self.__monomial_division(x, d) # may fail assert c # sanity check result += c dividend -= c * divisor dividend = dividend.canonical() # if `dividend` nonzero, try again # if `dividend` is zero, finish running = bool(dividend) break except ArithmeticError: # monomial division failed, try next one continue # inner loop else: # `dividend` exhausted, finish running = False pass if not hasattr(result, 'operator'): result = self.const(result) return result.canonical(), dividend.canonical() From python at mrabarnett.plus.com Sun May 12 12:48:54 2019 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 12 May 2019 17:48:54 +0100 Subject: [Python-ideas] Break multiple loop levels In-Reply-To: <20190512094444.GE5010@ando.pearwood.info> References: <4ffd4050-9f77-4970-41ab-e5e11fce115f@kynesim.co.uk> <7e1bd68e-68b1-e403-4ef4-fdc36ce8b757@interia.pl> <20190512091621.wyenlqovwx7kwu3m@phdru.name> <20190512094444.GE5010@ando.pearwood.info> Message-ID: <3a6afdab-fd48-acb0-3d02-a19eb755236b@mrabarnett.plus.com> On 2019-05-12 10:44, Steven D'Aprano wrote: > On Sun, May 12, 2019 at 11:16:21AM +0200, Oleg Broytman wrote: >> On Sun, May 12, 2019 at 01:36:28AM -0700, Elias Tarhini wrote: >> > If I may propose `break n` as a replacement for the original message's >> > `break break ... break`, where n>0 is the number of contiguous loops to >> > break out of and `break 1` is synonymous with `break`. Seems easier on my >> > eyes/noggin than counting out the individual `break` statements. >> >> This is very much error-prone because on any refactoring (increasing >> or decreasing the number of loop levels) one must increase/decrease all >> numbers in internal loops. >> >> Labels are at least stable. > > Indeed. > > Go has labels for loops: > > Label: > for { ... > break Label > } > > as does Rust: > > 'label: while condition: > { > ... > break label > } > > We can't use the same syntax in Python, but we might write it like this: > > @label while condition: > block > break label > > and similar with for loops: > > @outer for x in range(100): > @inner for y in range(3): > if x == y == 1: break # like ``break inner`` > if x == y == 2: break outer > print(x, y) > # break inner jumps to here > # break outer jumps to here > @ reminds me too much of decorators. How about this: