Mailman 3 April 2019 - Python-ideas

Start argument for itertools.accumulate() [Was: Proposal: A Reduce-Map Comprehension and a "last" builtin]
by Raymond Hettinger Oct. 23, 2019

Oct. 23, 2019

> On Friday, April 6, 2018 at 8:14:30 AM UTC-7, Guido van Rossum wrote: > On Fri, Apr 6, 2018 at 7:47 AM, Peter O'Connor <peter.ed...(a)gmail.com> wrote: >> So some more humble proposals would be: >> >> 1) An initializer to itertools.accumulate >> functools.reduce already has an initializer, I can't see any controversy to adding an initializer to itertools.accumulate > > See if that's accepted in the bug tracker. It did come-up once but was closed … [View More]for a number reasons including lack of use cases. However, Peter's signal processing example does sound interesting, so we could re-open the discussion. For those who want to think through the pluses and minuses, I've put together a Q&A as food for thought (see below). Everybody's design instincts are different -- I'm curious what you all think think about the proposal. Raymond --------------------------------------------- Q. Can it be done? A. Yes, it wouldn't be hard. _sentinel = object() def accumulate(iterable, func=operator.add, start=_sentinel): it = iter(iterable) if start is _sentinel: try: total = next(it) except StopIteration: return else: total = start yield total for element in it: total = func(total, element) yield total Q. Do other languages do it? A. Numpy, no. R, no. APL, no. Mathematica, no. Haskell, yes. * http://docs.scipy.org/doc/numpy/reference/generated/numpy.ufunc.accumulate.… * https://stat.ethz.ch/R-manual/R-devel/library/base/html/cumsum.html * http://microapl.com/apl/apl_concepts_chapter5.html \+ 1 2 3 4 5 1 3 6 10 15 * https://reference.wolfram.com/language/ref/Accumulate.html * https://www.haskell.org/hoogle/?hoogle=mapAccumL Q. How much work for a person to do it currently? A. Almost zero effort to write a simple helper function: myaccum = lambda it, func, start: accumulate(chain([start], it), func) Q. How common is the need? A. Rare. Q. Which would be better, a simple for-loop or a customized itertool? A. The itertool is shorter but more opaque (especially with respect to the argument order for the function call): result = [start] for x in iterable: y = func(result[-1], x) result.append(y) versus: result = list(accumulate(iterable, func, start=start)) Q. How readable is the proposed code? A. Look at the following code and ask yourself what it does: accumulate(range(4, 6), operator.mul, start=6) Now test your understanding: How many values are emitted? What is the first value emitted? Are the two sixes related? What is this code trying to accomplish? Q. Are there potential surprises or oddities? A. Is it readily apparent which of assertions will succeed? a1 = sum(range(10)) a2 = sum(range(10), 0) assert a1 == a2 a3 = functools.reduce(operator.add, range(10)) a4 = functools.reduce(operator.add, range(10), 0) assert a3 == a4 a4 = list(accumulate(range(10), operator.add)) a5 = list(accumulate(range(10), operator.add, start=0)) assert a5 == a6 Q. What did the Python 3.0 Whatsnew document have to say about reduce()? A. "Removed reduce(). Use functools.reduce() if you really need it; however, 99 percent of the time an explicit for loop is more readable." Q. What would this look like in real code? A. We have almost no real-world examples, but here is one from a StackExchange post: def wsieve(): # wheel-sieve, by Will Ness. ideone.com/mqO25A->0hIE89 wh11 = [ 2,4,2,4,6,2,6,4,2,4,6,6, 2,6,4,2,6,4,6,8,4,2,4,2, 4,8,6,4,6,2,4,6,2,6,6,4, 2,4,6,2,6,4,2,4,2,10,2,10] cs = accumulate(cycle(wh11), start=11) yield( next( cs)) # cf. ideone.com/WFv4f ps = wsieve() # codereview.stackexchange.com/q/92365/9064 p = next(ps) # 11 psq = p*p # 121 D = dict( zip( accumulate(wh11, start=0), count(0))) # start from sieve = {} for c in cs: if c in sieve: wheel = sieve.pop(c) for m in wheel: if not m in sieve: break sieve[m] = wheel # sieve[143] = wheel@187 elif c < psq: yield c else: # (c==psq) # map (p*) (roll wh from p) = roll (wh*p) from (p*p) x = [p*d for d in wh11] i = D[ (p-11) % 210] wheel = accumulate(cycle(x[i:] + x[:i]), start=psq) p = next(ps) ; psq = p*p next(wheel) ; m = next(wheel) sieve[m] = wheel [View Less]

17 49

support toml for pyproject support
by Jimmy Girardet July 29, 2019

July 29, 2019

Hi, I don't know if this was already debated but I don't know how to search in the whole archive of the list. For now the adoption of pyproject.toml file is more difficult because toml is not in the standard library. Each tool which wants to use pyproject.toml has to add a toml lib as a conditional or hard dependency. Since toml is now the standard configuration file format, it's strange the python does not support it in the stdlib lije it would have been strange to … [View More]

13 15

Re: [Python-ideas] Trigonometry in degrees
by Hugh Fisher May 8, 2019

May 8, 2019

> Date: Thu, 7 Jun 2018 12:33:29 +0000 > From: Robert Vanden Eynde <robertvandeneynde(a)hotmail.com> > To: python-ideas <python-ideas(a)python.org> > Subject: [Python-ideas] Trigonometry in degrees > Message-ID: > > > I suggest adding degrees version of the trigonometric functions in the math module. > > - Useful in Teaching and replacing calculators by python, importing something is seen by the young students much more easy than to define a … [View More]function. I agree that degrees are useful for teaching. They are also very useful for graphics programming, especially with my favourite OpenGL API. But I think that the use of radians in programming language APIs is more prevalent, so the initial advantage of easy learning will be outweighed by the long term inconvenience of adjusting to what everyone else is doing. Writing degrees(x) and radians(x) is a little inconvenient, but it does make it clear what units are being used. And even if your proposal is adopted, there is still going to be a lot of code around that uses the older math routines. With the current API it is a least safe to assume that angles are radians unless stated otherwise. > - Special values could be treated, aka when the angle is a multiple of 90, young students are often surprise to see that cos(pi/2) != 0 > > Testing for a special value Isn't very costly (x % 90 == 0) but it could be pointed out that there is a small overhead using the "degrees" equivalent of trig function because of the radians to degrees conversion And the special values testing. Not just young students :-) I agree with this, but I would prefer the check to be in the implementation of the existing functions as well. Any sin/cos very close to 0 becomes 0, any close to 1 becomes 1. > - Standard names will be chosen so that everyone will use the same name convention. I suggest adding a "d" like sind, cosd, tand, acosd, asind, atand, atan2d. Not "d". In the OpenGL 3D API, and many 3D languages/APIs since, appending "d" means "double precision". It's even sort of implied by the C math library which has sinf and friends for single precision. > > Creating a new package like 'from math.degrees import cos' however I would not recommend that because "cos" in the source code would mean to lookup the import to know if it's in degrees or radians (and that leads to very filthy bugs). Also "degrees" is already so the name would have to change the name of the package. Agree, not a good idea. -- cheers, Hugh Fisher [View Less]

16 29

Open parenthesis in REPL completion
by Danilo J. S. Bellini May 6, 2019

May 6, 2019

I'm not aware if that was already discussed, but something I find quite annoying is the REPL auto-complete that also includes the open parenthesis symbol. I think it shouldn't be included in TAB completion. At least twice a week I make mistakes like typing "help(something()" with unmatched parentheses, because the "something" was filled by a TAB completion, and the trailing open parenthesis wasn't expected (given that there's no such a completion elsewhere). -- Danilo J. S. Bellini -----------… [View More]

8 12

f-string "debug" conversion
by Eric V. Smith May 3, 2019

May 3, 2019

This idea was proposed to me at the core sprints last month by Larry Hastings. I've discussed it with a few people, who seem generally positive about it, and we've tweaked it a little bit. I've spent some time implementing it, and I think it's doable. I thought I'd post it here for any additional feedback. Here’s the idea: for f-strings, we add a !d conversion operator, which is superficially similar to !s, !r, and !a. The meaning of !d is: produce the text of the expression (not its … [View More]

12 36

Proposal: "?" Documentation Operator and easy reference to argument types/defaults/docstrings
by Peter O'Connor April 29, 2019

April 29, 2019

Dear all, Despite the general beauty of Python, I find myself constantly violating the "don't repeat yourself" maxim when trying to write clear, fully documented code. Take the following example: def func_1(a: int = 1, b: float = 2.5) -> float: """ Something about func_1 :param a: Something about param a :param b: Something else about param b :return: Something about return value of func_1 """ return a*b def func_2(c:float=3.… [View More]4, d: bool =True) -> float: """ Something about func_2 :param c: Something about param c :param d: Something else about param d :return: Something about return value """ return c if d else -c def main_function(a: int = 1, b: float = 2.5, d: bool = True) -> float: """ Something about main_function :param a: Something about param a :param b: Something else about param b :param d: Something else about param d :return: Something about return value """ return func_2(func_1(a=a, b=b), d=d) Which has the following problems: - Defaults are defined in multiple places, which very easily leads to bugs (I'm aware of **kwargs but it obfuscates function interfaces and usually does more harm than good) - Types are defined in multiple places - Documentation is copy-pasted when referencing a single thing from different places. (I can't count the number of types I've written ":param img: A (size_y, size_x, 3) RGB image" - I could now just reference a single RGB_IMAGE_DOC variable) - Argument names need to be written twice - in the header and documentation - and it's up to the user / IDE to make sure they stay in sync. I propose to resolve this with the following changes: - Argument/return documentation can be made inline with a new "?" operator. Documentation becomes a first class citizen. - Argument (type/default/doc) can be referenced by "func.args.<arg_name>.type" / "func.args.<arg_name>.default" / "func.args.<arg_name>.doc". Positional reference: e.g. "func.args[1].default" also allowed. If not specified, they take a special, built-in "Undefined" value (because None may have another meaning for defaults). Return type/doc can be referenced with "func.return.type" / "func.return.doc". This would result in the following syntax: def func_1( a: int = 1 ? 'Something about param a', b: float = 2.5 ? 'Something else about param b', ) -> float ? 'Something about return value of func_1': """ Something about func_1 """ return a*b def func_2( c: float=3.4 ? 'Something about param c', d: bool =True ? 'Something else about param d', ) -> float ? 'Something about return value': """ Something about func_2 """ return c if d else -c def main_function( a: func_1.args.a.type = func_1.args.a.default ? func_1.args.a.doc, b: func_1.args.b.type = func_1.args.b.default ? func_1.args.b.doc, d: func_2.args.d.type = func_2.args.d.default ? func_2.args.d.doc, ) -> func_2.return.type ? func2.return.doc: """ Something about main_function """ return func_2(func_1(a=a, b=b), d=d) If the main_function header seems repetitious (it does) we could allow for an optional shorthand notation like: def main_function( a :=? func_1.args.a, b :=? func_1.args.b, d :=? func_2.args.d, ) ->? func_2.return: """ Something about main_function """ return func_2(func_1(a=a, b=b), d=d) Where "a :=? func_1.args.a" means "argument 'a' takes the same type/default/documentation as argument 'a' of func_1". So what do you say? Yes it's a bold move, but I think in the long term it's badly needed. Perhaps something similar has been proposed already that I'm not aware of. [View Less]

6 7

Users of statistics software, what quantile functionality would be useful for you?
by Steven D'Aprano April 28, 2019

April 28, 2019

The statistics module is soon to get a quantile function. For those of you who use statistics software (whether in Python, or using other languages) and specifically use quantiles, what sort of functionality would be useful to you? For example: - evenly-spaced quantiles (say, at 0.25, 0.5, 0.75)? - unevenly-spaced quantiles (0.25, 0.8, 0.9, 0.995)? - one quantile at a time? - any specific definition? - quantiles of a distribution? - anything else? Thanks in advance. -- Steven

4 3

Idea: Allow multiple levels of tracers
by Ram Rachum April 28, 2019

April 28, 2019

Hi, Here's something I want in Python: Multiple levels of tracers working on top of each other, instead of just one. I'm talking about the tracer that one can set by calling sys.settrace. I've recently released PySnooper: https://github.com/cool-RR/PySnooper/ One of the difficulties I have, is that I can't debug or run the `coverage` tool on the core of this module. That's because the core is a trace function, and debuggers and coverage tools work by setting a trace function. When PySnooper … [View More]

5 12

Syntax to conditionally define a field in a dict
by Joshua Marshall April 27, 2019

April 27, 2019

Hello all, I have a use case where I need to send a `dict` to a module as an argument. Inside of this, it has a multi-level structure, but each field I need to set may only be set to a single value. Fields must be valid, non-empty strings. It looks a lot like the following in my code: ``` def my_func(val_1, val_2): return { "field_1": val_1, "next_depth": { "field_2": val_2 } } ``` What I want to do is: ``` def my_func(val_1, val_2): return {… [View More]

7 8

What are the strong use cases for str.rindex()?
by 林自均 April 26, 2019

April 26, 2019

Hi all, I found that there are str.index() and str.rindex(), but there is only list.index() and no list.rindex(). So I filed the issue https://bugs.python.org/issue36639 to provide list.rindex(). However, the issue was rejected and closed with the comment: > There were known, strong use cases for str.rindex(). The list.rindex() method was intentionally omitted. AFAICT no compelling use cases have arisen, so we should continue to leave it out. In general, we don't grow the core APIs … [View More]

7 11