Mailman 3 August 2021 - Python-ideas

Re: NAN handling in statistics functions
by Christopher Barker 01 Sep '21

01 Sep '21

On Tue, Aug 31, 2021 at 12:09 AM Stephen J. Turnbull < stephenjturnbull(a)gmail.com> wrote: > *sigh* __members__ is just the mechanism. As a consequence, Enums are > iterable, and they automatically DTRT with dir() and help() even if > there are no docstrings. but you didn't say dir(), you said __members__ ;-) But my bad, yes, dir() is easy and obvious, and works. As for help() -- for some reason, I never use it -- not sure why. I do use iPython's ?, which gives me: In [15]: NaNflag? Init signature: NaNflag( value, names=None, *, module=None, qualname=None, type=None, start=1, ) Docstring: An enumeration. File: /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/enum.py Type: EnumMeta Subclasses: which is not that useful, because the docstring is pretty worthless. help() on the other hand, is useful: class NaNflag(enum.Enum) | NaNflag(value, names=None, *, module=None, qualname=None, type=None, start=1 ) | | An enumeration. | | Method resolution order: | NaNflag | enum.Enum | builtins.object | | Data and other attributes defined here: | | Propogate = <NaNflag.Propogate: 3> | | Raise = <NaNflag.Raise: 1> | | Skip = <NaNflag.Skip: 2> | | ---------------------------------------------------------------------- | Data descriptors inherited from enum.Enum: | name | The name of the Enum member. | | value | The value of the Enum member. | | ---------------------------------------------------------------------- | Readonly properties inherited from enum.EnumMeta: | | __members__ | Returns a mapping of member name->value. | | This mapping lists all enum members, including aliases. Note that this | is a read-only view of the internal mapping. Hmm - maybe that's why I don't use help() much -- If an object has a decent docstring, I'd rather not see all that other cruft :-) So if you want a well documented Enum, you need to write a docstring, which is, or course, the case with everything in Python, so no problem there. (though a nice auto-generated docstring would be good -- maybe a feature request for the future) I've been convinced, Enums do provide a nice way to document flag options. But unfortunately, it doesn't put that documentation where I'd want it -- in the docstring of the functions that use it. DRY is hard with documentation -- this reminds me of when there's a big class hierarchy, and you have go read the docs way up the tree to know how to use a class. Now show me how to do any of those with str > codes. > In that case, the documentation needs to go in the docstring of the function where they are used, which is where I want it anyway. That's a documentation style matter, though. Since just about every > function in the statistics package will take the 'nans' argument, not > only the writer, but most readers of the documentation will favor > DRYing it once, IMHO YMMV. > I don't think readers will have a preference for having something documented in one place that is not the place you actually use it. They WILL appreciate that all functions in the statistics package work the same way though, that's for sure. Anyway, I was expressing a preference, and I still have that preference, but Enums are a fine option as well. And the proposed idea of using the string value for the Enum, so that users can use either has its appeal. For me, it's like Lays potato chips; I can't call just one. Almost > all my use cases call for mean, standard deviation, and median. > hmm, another topic, but computing mean and standard deviation at the same time would make sense. Bringing back to the decision Steven needs to make: I still prefer string flags, but there are certainly good arguments to be made for an Enum. If you do go with an Enum, I suggest: - The flag names are put in the statistics module namespace. - The Enum gets a good docstring. - The functions that use the Enum also document the flags and what they mean. - to keep that dry, you could automatically insert the Enum docstring into each function's docstring, so that it would be there where users need it. That's my $ 0.2 -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

1 0

Re: NAN handling in statistics functions
by Christopher Barker 31 Aug '21

31 Aug '21

On Mon, Aug 30, 2021 at 10:22 AM Stephen J. Turnbull < stephenjturnbull(a)gmail.com> wrote: > Christopher Barker writes: > > > e.g.: what are the valid values? > > That's easy: MyEnum.__members__. > Seriously? you are arguing that Enums are better because they are self documenting, when you have to poke at a dunder to get the information ?!? I'm actually kind of surprised there isn't an obvious way to check that. 2. The class attribute __doc__ is treated specially: it does not > become an Enum member, and it is treated as you would expect by > help(). OK, so this is one advantage (for the use case at hand) you can put the docs for what the NaNFlags are in teh Enum docstring, and then it is documented for all function that use that flag. However, this is helpful (and DRY) for the author of the package -- but I think less useful for the users of the package. I want to (in iPython) do: statistics.median? and see everything I need to know to use it, not get a reference to an Enum that I then need to go look up. > But what they do is create a burden of extra code to read and write. > I think more likely it would be > > from statistics import median > from statistics import NANPolicy as NP > > result = median(the_data, nans=NP.OMIT) > Exactly -- this is a fair bit more awkward than: from statistics import median result = median(the_data, nans="omit") I suppose less so if you are using more than a couple calls to statistics functions, but how common is that? Python 3.11 Enums have the global_enum decorator, which injects the > members into the global namespace. Then the above could be: > > from statistics import median > from statistics.nanpolicy import * > > result = median(the_data, nans=OMIT) > hmm, I suppose I'd follow numpy tradition, and do: import statistics as st result = st.median(the_data, nans=st.OMIT) but that only makes sense for numpy because we tend to use a LOT of numpy once we are using any :-) But: if an ENUM is used, do please put all the flags in the statistics namespace. -CHB Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

4 4

synatx sugar for quickly run shell command and return stdout of shell command as string result
by Evan Greenup 30 Aug '21

30 Aug '21

Currently, when what to execute external command, either os.system() or subprocess functions should be invoked. However, it is not so convenient, it would be nice to use "``" symbol to brace the shell command inside. like: stdout_result_str = `cat file.txt | sort | grep hello` Its is enhanced feature which combine both subprocess and os.system Because, subprocess can return output of file, but does not support pipe in command. While os.system can apply pipe in command but doesn't support return stdout result. This can be very convenient to make python available for some build script which ruby now do well like homebrew ... simply `echo hello | wc -l` while only execute command and the result will just like other unassigned expression, the result will be finally discarded. There won't be big difficulties to implement.

11 11

Re: NAN handling in statistics functions
by David Mertz, Ph.D. 29 Aug '21

29 Aug '21

I was thinking of the Cauchy distribution, with undefined variance. But Augustin-Louis Cauchy had quite a few things named after him. I know best Cauchy sequences as a construction of Real numbers. On Sun, Aug 29, 2021, 2:36 AM Stephen J. Turnbull < stephenjturnbull(a)gmail.com> wrote: > David Mertz, Ph.D. writes: > > > > I have a distribution for you: Cauchy. :-) > > > > > > > Oh? Because NaN is the *result* of `stats.variance(cauchy)`? > > > > It still seems like as INPUTS to a stats function NaN ~= missing. > > Well, that was a little opaque. I wanted to fit it into the "I have a > ... for you" pattern. > > What I meant by "Cauchy" was just a ratio whose denominator's support > includes zero. It's occasionally seen in (not so hot) dynamic > economic trade models (designed on the assumption of positive growth > rates oops :-�)P). I don't know about other contexts. > > Steve > >

1 0

Re: NAN handling in statistics functions
by David Mertz, Ph.D. 29 Aug '21

29 Aug '21

On Sat, Aug 28, 2021, 8:34 AM Stephen J. Turnbull < stephenjturnbull(a)gmail.com> wrote: > David Mertz, Ph.D. writes: > > > NANs do not necessarily represent missing data. > > > I think in the context of `stats` they do. But this is color of > bikeshed, and I defer to you, of course. > > I have a distribution for you: Cauchy. :-) > Oh? Because NaN is the *result* of `stats.variance(cauchy)`? It still seems like as INPUTS to a stats function NaN ~= missing.

1 0

Complete recursive pickle dump
by Evan Greenup 28 Aug '21

28 Aug '21

Currently, pickle can only save some very simple data types into bytes. And the object itself contains reference to some builtin data or variable, only reference will be saved. I am wondering if it is possible recursively persist all the data into pickle representation. Even if some data might be builtin function of variable, it will represent it in very raw (memory level). If it contains function it will persist all its attribute, and code object. Method-wrapper and all its attribute will be persist entirely. If it contains two attribute with same memory location `id(x)` is same, then they can be persist as one, and reference each other. For example, if user pickle recursively dump a numpy ndarray into a pickle file, when user pickle load this file from a system which doesn't install numpy, its method should all works properly like transpose, reshape ... When applying recursively dump, the output pickle file or bytes might be very large. All related memory data will be dumped. As the example of numpy ndarray, when doing recursively dump, large part of numpy library's content will be persist except those really lose or none coupled part. This is actually a cool feature, because pickle is to keep python object, completeness is a great advantage of pickle than json and other data persist library. Sent with ProtonMail Secure Email.

2 1

Re: We should have an explicit concept of emptiness for collections
by Finn Mason 26 Aug '21

26 Aug '21

Tim, you're right. I'm sorry. When this was first posted, I was sure it would be resolved quickly and easily. I very badly communicated that. On Thu, Aug 26, 2021, 12:59 AM Tim Hoffmann <tim.hoffmann(a)mailbox.org> wrote: > Hi Finn, > > as a personal feedback: I perceive this comment as disrespectful and not > welcoming. I've been trying to make a constructive contribution and this > comment comes across as "Hooray we got rid of that/him. And anyway the > suggestion was nonsensical. Why have we been talking about it at all?". > It's hard to read context from a short mail. Maybe I'm misinterperting. If > not, it's still your right to feel that way, but please reconsider if it's > necessary to post such comments. > > Tim > > Finn Mason <finnjavier08(a)gmail.com> hat am 26.08.2021 01:50 geschrieben: > > > Yay, it's over! > (I can't believe it got this much attention.) > > On Wed, Aug 25, 2021, 3:19 PM Tim Hoffmann via Python-ideas < > python-ideas(a)python.org> wrote: > > Ok, I have no problem ignoring PEP-8 where it's not applicable. I've > brought this topic up, because I thought there could be a improvement > either in PEP-8 and/or by adding something to the language. I still do > think that, but I accept that the problem it solves is not considered > relevant enough here to take further action. > > Anyway, thanks all for the discussion! > > Tim > _______________________________________________ > Python-ideas mailing list -- python-ideas(a)python.org > To unsubscribe send an email to python-ideas-leave(a)python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/HXW4I… > Code of Conduct: http://python.org/psf/codeofconduct/ > > _______________________________________________ > Python-ideas mailing list -- python-ideas(a)python.org > To unsubscribe send an email to python-ideas-leave(a)python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/Z5DS5… > Code of Conduct: http://python.org/psf/codeofconduct/ > >

1 0

Add explicit method overloading to STL
by Will Bradley 26 Aug '21

26 Aug '21

Don't know if this is already a PEP, but I'd love to see something like this <https://www.codementor.io/@arpitbhayani/overload-functions-in-python-13e32a…> in Python— a decorator @overload that creates multiple copies of functions/methods based on their arguments' types. (This is much narrower in scope than PEP 3124, before anyone asks.)

5 6

We should have an explicit concept of emptiness for collections
by Tim Hoffmann 26 Aug '21

26 Aug '21

Hi all, The Programming Recommendations section in PEP-8 states "For sequences, (strings, lists, tuples), use the fact that empty sequences are false:" # Correct: if not seq: if seq: # Wrong: if len(seq): if not len(seq): In the talk "When Python Practices Go Wrong" Brandon Rhodes makes a good point against this practice based on "explicit is better than implicit" (https://youtu.be/S0No2zSJmks?t=873). He advertizes using if len(seq): While that is as explicit as one can get within the current language, it could still be more explicit: Semantically, we're not interested in the (zero) length of the sequence, but want to know if it is empty. **Proposal** Therefore, I propose syntax for an explicit empty check if isempty(seq): (i) or if seq.is_empty() (ii) This proposal is mainly motivated by the Zen verses "Explicit is better than implicit" and "Readability counts". The two variants have slightly different advantages and disadvantages: (i) would add an __isempty__ protocol and a corresponding isempty() builtin function. One could argue that this is somewhat redundant with __len__. However, this is a typical pattern in collection.abc abstract base classes: There are only a few abstract methods and futher concepts are added as mixin methods (which are by default implemented using the abstract methods). https://docs.python.org/3/library/collections.abc.html#collections-abstract… (ii) would alternatively only implement a method on the collection. There's also precedence for predefined methods on collections, e.g. Sequence.index(). Advantages over the protocol approach are: - It's a smaller language change than adding a protocol and a builtin function - The order seq.is_empty() matches the order used in spoken english ("if the sequence is empty") which is more readable than isempty(seq). - It's tab-completable (-> usability) Disadvantages: - Emptiness is similar to length, and people might be used to the builtin-method concept for such things. - A protocol is more powerful than a method: One can support isempty() via len() even for objects that do not implement __isempty__. However, this more a theoretical advantage. I assume that in practice all collections derive from the collection.abc classes, so if the is_empty() method is implemented there, all typical use cases should be covered. I'm looking forward to your feedback! Tim

19 57

Re: We should have an explicit concept of emptiness for collections
by Christopher Barker 25 Aug '21

25 Aug '21

Bringing this back on list: On Tue, Aug 24, 2021 at 9:58 PM David Mertz, Ph.D. <david.mertz(a)gmail.com> wrote: > Sorry, I should have been more explicit. The several kinda of queues can > all "contain" items, but do not respond to len(). > yeah, I should have looked more closely at your list Though i would consider that an oversight, len(a_queue) could be handy. There is qsize() -- I wonder if there's a reason not to have __len__ do the same thing -- O(n) maybe? See Guido's point that there's an assumption tha len() will be O(1). This is an argument for the OP's point -- there is no standard way to check emptiness. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

3 2