Python-ideas August 2021

python-ideas@python.org

73 participants
41 discussions

Python Documentation in Bengali (translation)

by sharifmehedi24＠outlook.com

Hi there all, I am preparing a team to start translating python documentation to Bengali. I'd appreciate some reference materials, comments and ideas about this. Thank you

5 days, 23 hours

Add check_methods function to standard library

by Finn Mason

Hello, In _collections_abc.py is a private function titled `_check_methods`. It takes a class and a number of method names (as strings), checks if the class has all of the methods, and returns NotImplemented if any are missing. The code is below: ``` def _check_methods(C, *methods): mro = C.__mro__ for method in methods: for B in mro: if method in B.__dict__: if B.__dict__[method] is None: return NotImplemented break else: return NotImplemented return True ``` This is an incredibly convenient function (referred to as check_methods here on out) for creating abstract base classes, and is much simpler than using `hasattr` for each method you want to check. For example: ``` >>> from abc import ABCMeta >>> # Without check_methods >>> class A(metaclass=ABCMeta): ... @classmethod ... def __subclasshook__(cls, subclass): ... return (hasattr(subclass, 'foo') and ... callable(subclass.foo) and ... hasattr(subclass, 'bar') and ... callable(subclass.bar) or ... NotImplemented) ... >>> # With check_methods >>> class B(metaclass=ABCMeta): ... @classmethod ... def __subclasshook(cls, subclass): ... return check_methods(subclass, 'foo', 'bar') ... >>> ``` This would be a great function to add to the standard lib, perhaps in the `abc` module. One problem with `check_methods` as defined in _collections_abc.py is that it doesn't check if the name is callable. Also, type hints and more readable variables may be desirable. The final code, if implemented, may look something like this: ``` # In imports section: from typing import Literal def check_methods(Class: type, *methods: str) -> Literal[True, NotImplemented]: """Check if class `Class` has methods `methods`.""" mro = Class.__mro__ for method in methods: for Base in mro: if (attr := getattr(Base, method, None)) is not None: if not callable(attr): return NotImplemented break else: return NotImplemented return True ``` Again, this would be a great function to add to the `abc` module or a similar one. I've proposed this in issue 44941. Another possible implementation may be similar to a dataclass that implements __subclasshook__ for you with the desired method checks. --Finn

1 week, 5 days

Additional Unpacking Generalizations for assignment

by 笹原康央

Hello developers. I like star expression. It would be even better if we could extend the idea of PEP448 to provide the following syntax at the time of assignment. ``` python a, b, d, **_ = **{"a":0, " b":1, "c":2}, d=3 ``` Ideally, it would be nice to be able to provide the same syntax when calling the function. I think this applies to the zen of python called "There should be one-- and preferably only one --obvious way to do it.". ``` python a, *args, b, **kwargs = 1, 2, *[3,4], **{"b", 4}, c=4 ( a: str, *args, b: list = [], **kwargs ) = 1, 2, *[3,4], **{"b", 4}, c=4 func = lambda( a: str, *args, b: list = [], **kwargs ): print(a, b) ``` But it would be difficult because it would conflict with various syntaxes(etc. PEP 3132). We may have to declare it explicitly to avoid confusion. For example, prepare a keyword for unpack the positional arguments and keyword arguments at the same time. ``` a, *args, b, **kwargs = ***(1, 2, *[3,4], **{"b", 4}, c=4) ``` You can now explicitly recognize that you are using the behavior of the new assignment. I hope this is an idea to develop python. Thanks.

1 week, 6 days

PEP8 mandatory-is rule

by Nick Parlante

Hi there python-ideas - I've been teaching Python as a first programming language for a few years, and from that experience I want to propose a change to PEP8. I'm sure the default position for PEP8 is to avoid changing it. However, for this one rule I think a good case can be made to make it optional, so let me know what you think. Let me start with what I've learned from teaching students in Java and now in Python. In Java, you use == for ints, but you need to use equals() for strings. Of course students screw this up constantly, using == in a context that calls for equals() and their code does not work right. Then for Java arrays a different comparison function is required, and so it goes. To teach comparisons in Python, I simply say "just use ==" - it works for ints, for strings, even for lists. Students are blown away by how nice and simple this is. This is how things should work. Python really gets this right. So what is the problem? The problem for Python is what I will call the "mandatory-is" rule in PEP8, which reads: Comparisons to singletons like None should always be done with is or is not, never the equality operators. For the students, this comes up in the first week of the course with lines like "if x == None:" which work perfectly with == but should use is/is-not for PEP8 conformance. My guess is that this rule is in PEP8 because, within a Python implementation, it is within the programmer's mental model that, say, False is a singleton. The mandatory-is rule is in PEP8 to reinforce that mental model by requiring the is operator. Plus it probably runs a tiny bit faster. However, for "regular" Python code, not implementing Python, forcing the use of is instead of the simpler == is unneeded and unhelpful (and analogously forcing "is not" when != works correctly). What is the benefit of forcing the is operator there? I would say it spreads an awareness of the details of how certain values are allocated within Python. That's not much of a benefit, and it's kind of circular. Like if programmers were permitted to use ==, they wouldn't need to know the details of how Python allocates those values. Being shielded from implementation details is a Python strength - think of the Java vs. Python story above. Is Java better because it builds an awareness in the programmer of the different comparison functions for different types? Of course not! Python is better in that case because it lets the programmer simply use == and not think about those details. Understanding the singleton strategy is important in some corners of coding, but forcing the is operator on all Python code is way out of proportion to the benefit. As a practical matter, the way this comes up for my students is that IDEs by default will put warning marks around PEP8 violations in their code. Mostly this IDE-coaching is very helpful for students learning Python. For example, It's great that beginning Python programmers learn to put one space around operators right from the first day. Having taught thousands of introductory Python students, the one PEP8 rule that causes problems is this mandatory-is rule. As a teacher, this is especially jarring since the "just use ==" rule is so effortless to use correctly. In contrast, the mandatory-is rule adds a little pause where the programmer should think about which comparison operator is the correct one to use. It's not hard, but it feels unnecessary. As a contrasting example, in the language C, programmers need to understand == vs. is right from the first day. You can't get anything done in C without understanding that distinction. However that is just not true for regular (not-Python-implementation) Python code, where == works correctly for the great majority of cases. Here is my proposal: Add the following parenthetical to the mandatory-is rule: (this rule is optional for code that is not part of an implementation of Python). So in effect, programmers outside of a Python implementation can choose to use == or is for the "if x == None:" case. In this way, PEP8 conforming code before the change is still conforming. Moving forward, I would expect that regular code will trend towards using == in such a case, reserving is for the rare cases where it is needed for correctness. PEP8 was originally just for Python implementations, so why is this change needed? Because as a practical matter, the vast majority of code that is using PEP8 is not part of a Python implementation. This may not have been the original mission of PEP8, but it is how things have worked out. Now we are in a situation where the rules in PEP8 are sent out to this ocean of Python programmers of many different ability levels writing regular code that is not a Python implementation. One could imagine a separate PEP800 style guide for regular code, but we don't need to do that, because in almost all cases PEP8 works great for regular code. I have taught thousands of new Python programmers, and the only place where PEP8 serves them poorly is this mandatory-is rule. Therefore instead of a separate style guide for regular code, I propose an exception for this one problem rule. Ultimately this comes down to the question - should PEP8 push regular, not-Python-implementation code to use is for singletons in cases where == works perfectly? Seeing how effortless it is for programmers to use == as their first choice, I think PEP8 should allow that practice. Best, Nick

2 weeks, 4 days

NAN handling in statistics functions

by Steven D'Aprano

At the moment, the handling of NANs in the statistics module is implementation dependent. In practice, that *usually* means that if your data has a NAN in it, the result you get will probably be a NAN. >>> statistics.mean([1, 2, float('nan'), 4]) nan But there are unfortunate exceptions to this: >>> statistics.median([1, 2, float('nan'), 4]) nan >>> statistics.median([float('nan'), 1, 2, 4]) 1.5 I've spoken to users of other statistics packages and languages, such as R, and I cannot find any consensus on what the "right" behaviour should be for NANs except "not that!". So I propose that statistics functions gain a keyword only parameter to specify the desired behaviour when a NAN is found: - raise an exception - return NAN - ignore it (filter out NANs) which seem to be the three most common preference. (It seems to be split roughly equally between the three.) Thoughts? Objections? Does anyone have any strong feelings about what should be the default? -- Steve

3 weeks

Re: NAN handling in statistics functions

by Christopher Barker

On Tue, Aug 31, 2021 at 12:09 AM Stephen J. Turnbull < stephenjturnbull(a)gmail.com> wrote: > *sigh* __members__ is just the mechanism. As a consequence, Enums are > iterable, and they automatically DTRT with dir() and help() even if > there are no docstrings. but you didn't say dir(), you said __members__ ;-) But my bad, yes, dir() is easy and obvious, and works. As for help() -- for some reason, I never use it -- not sure why. I do use iPython's ?, which gives me: In [15]: NaNflag? Init signature: NaNflag( value, names=None, *, module=None, qualname=None, type=None, start=1, ) Docstring: An enumeration. File: /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/enum.py Type: EnumMeta Subclasses: which is not that useful, because the docstring is pretty worthless. help() on the other hand, is useful: class NaNflag(enum.Enum) | NaNflag(value, names=None, *, module=None, qualname=None, type=None, start=1 ) | | An enumeration. | | Method resolution order: | NaNflag | enum.Enum | builtins.object | | Data and other attributes defined here: | | Propogate = <NaNflag.Propogate: 3> | | Raise = <NaNflag.Raise: 1> | | Skip = <NaNflag.Skip: 2> | | ---------------------------------------------------------------------- | Data descriptors inherited from enum.Enum: | name | The name of the Enum member. | | value | The value of the Enum member. | | ---------------------------------------------------------------------- | Readonly properties inherited from enum.EnumMeta: | | __members__ | Returns a mapping of member name->value. | | This mapping lists all enum members, including aliases. Note that this | is a read-only view of the internal mapping. Hmm - maybe that's why I don't use help() much -- If an object has a decent docstring, I'd rather not see all that other cruft :-) So if you want a well documented Enum, you need to write a docstring, which is, or course, the case with everything in Python, so no problem there. (though a nice auto-generated docstring would be good -- maybe a feature request for the future) I've been convinced, Enums do provide a nice way to document flag options. But unfortunately, it doesn't put that documentation where I'd want it -- in the docstring of the functions that use it. DRY is hard with documentation -- this reminds me of when there's a big class hierarchy, and you have go read the docs way up the tree to know how to use a class. Now show me how to do any of those with str > codes. > In that case, the documentation needs to go in the docstring of the function where they are used, which is where I want it anyway. That's a documentation style matter, though. Since just about every > function in the statistics package will take the 'nans' argument, not > only the writer, but most readers of the documentation will favor > DRYing it once, IMHO YMMV. > I don't think readers will have a preference for having something documented in one place that is not the place you actually use it. They WILL appreciate that all functions in the statistics package work the same way though, that's for sure. Anyway, I was expressing a preference, and I still have that preference, but Enums are a fine option as well. And the proposed idea of using the string value for the Enum, so that users can use either has its appeal. For me, it's like Lays potato chips; I can't call just one. Almost > all my use cases call for mean, standard deviation, and median. > hmm, another topic, but computing mean and standard deviation at the same time would make sense. Bringing back to the decision Steven needs to make: I still prefer string flags, but there are certainly good arguments to be made for an Enum. If you do go with an Enum, I suggest: - The flag names are put in the statistics module namespace. - The Enum gets a good docstring. - The functions that use the Enum also document the flags and what they mean. - to keep that dry, you could automatically insert the Enum docstring into each function's docstring, so that it would be there where users need it. That's my $ 0.2 -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

3 weeks

Re: NAN handling in statistics functions

by Christopher Barker

On Mon, Aug 30, 2021 at 10:22 AM Stephen J. Turnbull < stephenjturnbull(a)gmail.com> wrote: > Christopher Barker writes: > > > e.g.: what are the valid values? > > That's easy: MyEnum.__members__. > Seriously? you are arguing that Enums are better because they are self documenting, when you have to poke at a dunder to get the information ?!? I'm actually kind of surprised there isn't an obvious way to check that. 2. The class attribute __doc__ is treated specially: it does not > become an Enum member, and it is treated as you would expect by > help(). OK, so this is one advantage (for the use case at hand) you can put the docs for what the NaNFlags are in teh Enum docstring, and then it is documented for all function that use that flag. However, this is helpful (and DRY) for the author of the package -- but I think less useful for the users of the package. I want to (in iPython) do: statistics.median? and see everything I need to know to use it, not get a reference to an Enum that I then need to go look up. > But what they do is create a burden of extra code to read and write. > I think more likely it would be > > from statistics import median > from statistics import NANPolicy as NP > > result = median(the_data, nans=NP.OMIT) > Exactly -- this is a fair bit more awkward than: from statistics import median result = median(the_data, nans="omit") I suppose less so if you are using more than a couple calls to statistics functions, but how common is that? Python 3.11 Enums have the global_enum decorator, which injects the > members into the global namespace. Then the above could be: > > from statistics import median > from statistics.nanpolicy import * > > result = median(the_data, nans=OMIT) > hmm, I suppose I'd follow numpy tradition, and do: import statistics as st result = st.median(the_data, nans=st.OMIT) but that only makes sense for numpy because we tend to use a LOT of numpy once we are using any :-) But: if an ENUM is used, do please put all the flags in the statistics namespace. -CHB Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

3 weeks

synatx sugar for quickly run shell command and return stdout of shell command as string result

by Evan Greenup

Currently, when what to execute external command, either os.system() or subprocess functions should be invoked. However, it is not so convenient, it would be nice to use "``" symbol to brace the shell command inside. like: stdout_result_str = `cat file.txt | sort | grep hello` Its is enhanced feature which combine both subprocess and os.system Because, subprocess can return output of file, but does not support pipe in command. While os.system can apply pipe in command but doesn't support return stdout result. This can be very convenient to make python available for some build script which ruby now do well like homebrew ... simply `echo hello | wc -l` while only execute command and the result will just like other unassigned expression, the result will be finally discarded. There won't be big difficulties to implement.

3 weeks, 2 days

Re: NAN handling in statistics functions

by David Mertz, Ph.D.

I was thinking of the Cauchy distribution, with undefined variance. But Augustin-Louis Cauchy had quite a few things named after him. I know best Cauchy sequences as a construction of Real numbers. On Sun, Aug 29, 2021, 2:36 AM Stephen J. Turnbull < stephenjturnbull(a)gmail.com> wrote: > David Mertz, Ph.D. writes: > > > > I have a distribution for you: Cauchy. :-) > > > > > > > Oh? Because NaN is the *result* of `stats.variance(cauchy)`? > > > > It still seems like as INPUTS to a stats function NaN ~= missing. > > Well, that was a little opaque. I wanted to fit it into the "I have a > ... for you" pattern. > > What I meant by "Cauchy" was just a ratio whose denominator's support > includes zero. It's occasionally seen in (not so hot) dynamic > economic trade models (designed on the assumption of positive growth > rates oops :-�)P). I don't know about other contexts. > > Steve > >

3 weeks, 3 days

Re: NAN handling in statistics functions

by David Mertz, Ph.D.

On Sat, Aug 28, 2021, 8:34 AM Stephen J. Turnbull < stephenjturnbull(a)gmail.com> wrote: > David Mertz, Ph.D. writes: > > > NANs do not necessarily represent missing data. > > > I think in the context of `stats` they do. But this is color of > bikeshed, and I defer to you, of course. > > I have a distribution for you: Cauchy. :-) > Oh? Because NaN is the *result* of `stats.variance(cauchy)`? It still seems like as INPUTS to a stats function NaN ~= missing.

3 weeks, 3 days

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas August 2021