Mailman 3 April 2019 - Python-ideas

Start argument for itertools.accumulate() [Was: Proposal: A Reduce-Map Comprehension and a "last" builtin]
by Raymond Hettinger 23 Oct '19

23 Oct '19

> On Friday, April 6, 2018 at 8:14:30 AM UTC-7, Guido van Rossum wrote: > On Fri, Apr 6, 2018 at 7:47 AM, Peter O'Connor <peter.ed...(a)gmail.com> wrote: >> So some more humble proposals would be: >> >> 1) An initializer to itertools.accumulate >> functools.reduce already has an initializer, I can't see any controversy to adding an initializer to itertools.accumulate > > See if that's accepted in the bug tracker. It did come-up once but was closed for a number reasons including lack of use cases. However, Peter's signal processing example does sound interesting, so we could re-open the discussion. For those who want to think through the pluses and minuses, I've put together a Q&A as food for thought (see below). Everybody's design instincts are different -- I'm curious what you all think think about the proposal. Raymond --------------------------------------------- Q. Can it be done? A. Yes, it wouldn't be hard. _sentinel = object() def accumulate(iterable, func=operator.add, start=_sentinel): it = iter(iterable) if start is _sentinel: try: total = next(it) except StopIteration: return else: total = start yield total for element in it: total = func(total, element) yield total Q. Do other languages do it? A. Numpy, no. R, no. APL, no. Mathematica, no. Haskell, yes. * http://docs.scipy.org/doc/numpy/reference/generated/numpy.ufunc.accumulate.… * https://stat.ethz.ch/R-manual/R-devel/library/base/html/cumsum.html * http://microapl.com/apl/apl_concepts_chapter5.html \+ 1 2 3 4 5 1 3 6 10 15 * https://reference.wolfram.com/language/ref/Accumulate.html * https://www.haskell.org/hoogle/?hoogle=mapAccumL Q. How much work for a person to do it currently? A. Almost zero effort to write a simple helper function: myaccum = lambda it, func, start: accumulate(chain([start], it), func) Q. How common is the need? A. Rare. Q. Which would be better, a simple for-loop or a customized itertool? A. The itertool is shorter but more opaque (especially with respect to the argument order for the function call): result = [start] for x in iterable: y = func(result[-1], x) result.append(y) versus: result = list(accumulate(iterable, func, start=start)) Q. How readable is the proposed code? A. Look at the following code and ask yourself what it does: accumulate(range(4, 6), operator.mul, start=6) Now test your understanding: How many values are emitted? What is the first value emitted? Are the two sixes related? What is this code trying to accomplish? Q. Are there potential surprises or oddities? A. Is it readily apparent which of assertions will succeed? a1 = sum(range(10)) a2 = sum(range(10), 0) assert a1 == a2 a3 = functools.reduce(operator.add, range(10)) a4 = functools.reduce(operator.add, range(10), 0) assert a3 == a4 a4 = list(accumulate(range(10), operator.add)) a5 = list(accumulate(range(10), operator.add, start=0)) assert a5 == a6 Q. What did the Python 3.0 Whatsnew document have to say about reduce()? A. "Removed reduce(). Use functools.reduce() if you really need it; however, 99 percent of the time an explicit for loop is more readable." Q. What would this look like in real code? A. We have almost no real-world examples, but here is one from a StackExchange post: def wsieve(): # wheel-sieve, by Will Ness. ideone.com/mqO25A->0hIE89 wh11 = [ 2,4,2,4,6,2,6,4,2,4,6,6, 2,6,4,2,6,4,6,8,4,2,4,2, 4,8,6,4,6,2,4,6,2,6,6,4, 2,4,6,2,6,4,2,4,2,10,2,10] cs = accumulate(cycle(wh11), start=11) yield( next( cs)) # cf. ideone.com/WFv4f ps = wsieve() # codereview.stackexchange.com/q/92365/9064 p = next(ps) # 11 psq = p*p # 121 D = dict( zip( accumulate(wh11, start=0), count(0))) # start from sieve = {} for c in cs: if c in sieve: wheel = sieve.pop(c) for m in wheel: if not m in sieve: break sieve[m] = wheel # sieve[143] = wheel@187 elif c < psq: yield c else: # (c==psq) # map (p*) (roll wh from p) = roll (wh*p) from (p*p) x = [p*d for d in wh11] i = D[ (p-11) % 210] wheel = accumulate(cycle(x[i:] + x[:i]), start=psq) p = next(ps) ; psq = p*p next(wheel) ; m = next(wheel) sieve[m] = wheel

17 49

support toml for pyproject support
by Jimmy Girardet 30 Jul '19

30 Jul '19

Hi, I don't know if this was already debated but I don't know how to search in the whole archive of the list. For now the adoption of pyproject.toml file is more difficult because toml is not in the standard library. Each tool which wants to use pyproject.toml has to add a toml lib as a conditional or hard dependency. Since toml is now the standard configuration file format, it's strange the python does not support it in the stdlib lije it would have been strange to not have the configparser module. I know it's complicated to add more and more thing to the stdlib but I really think it is necessary for python packaging being more consistent. Maybe we could thought to a readonly lib to limit the added code. If it's conceivable, I'd be happy to help in it. Nice Day guys and girls. Jimmy

13 15

Re: [Python-ideas] Trigonometry in degrees
by Hugh Fisher 08 May '19

08 May '19

> Date: Thu, 7 Jun 2018 12:33:29 +0000 > From: Robert Vanden Eynde <robertvandeneynde(a)hotmail.com> > To: python-ideas <python-ideas(a)python.org> > Subject: [Python-ideas] Trigonometry in degrees > Message-ID: > > > I suggest adding degrees version of the trigonometric functions in the math module. > > - Useful in Teaching and replacing calculators by python, importing something is seen by the young students much more easy than to define a function. I agree that degrees are useful for teaching. They are also very useful for graphics programming, especially with my favourite OpenGL API. But I think that the use of radians in programming language APIs is more prevalent, so the initial advantage of easy learning will be outweighed by the long term inconvenience of adjusting to what everyone else is doing. Writing degrees(x) and radians(x) is a little inconvenient, but it does make it clear what units are being used. And even if your proposal is adopted, there is still going to be a lot of code around that uses the older math routines. With the current API it is a least safe to assume that angles are radians unless stated otherwise. > - Special values could be treated, aka when the angle is a multiple of 90, young students are often surprise to see that cos(pi/2) != 0 > > Testing for a special value Isn't very costly (x % 90 == 0) but it could be pointed out that there is a small overhead using the "degrees" equivalent of trig function because of the radians to degrees conversion And the special values testing. Not just young students :-) I agree with this, but I would prefer the check to be in the implementation of the existing functions as well. Any sin/cos very close to 0 becomes 0, any close to 1 becomes 1. > - Standard names will be chosen so that everyone will use the same name convention. I suggest adding a "d" like sind, cosd, tand, acosd, asind, atand, atan2d. Not "d". In the OpenGL 3D API, and many 3D languages/APIs since, appending "d" means "double precision". It's even sort of implied by the C math library which has sinf and friends for single precision. > > Creating a new package like 'from math.degrees import cos' however I would not recommend that because "cos" in the source code would mean to lookup the import to know if it's in degrees or radians (and that leads to very filthy bugs). Also "degrees" is already so the name would have to change the name of the package. Agree, not a good idea. -- cheers, Hugh Fisher

16 29

Open parenthesis in REPL completion
by Danilo J. S. Bellini 07 May '19

07 May '19

I'm not aware if that was already discussed, but something I find quite annoying is the REPL auto-complete that also includes the open parenthesis symbol. I think it shouldn't be included in TAB completion. At least twice a week I make mistakes like typing "help(something()" with unmatched parentheses, because the "something" was filled by a TAB completion, and the trailing open parenthesis wasn't expected (given that there's no such a completion elsewhere). -- Danilo J. S. Bellini --------------- "*It is not our business to set up prohibitions, but to arrive at conventions.*" (R. Carnap)

8 12

f-string "debug" conversion
by Eric V. Smith 03 May '19

03 May '19

This idea was proposed to me at the core sprints last month by Larry Hastings. I've discussed it with a few people, who seem generally positive about it, and we've tweaked it a little bit. I've spent some time implementing it, and I think it's doable. I thought I'd post it here for any additional feedback. Here’s the idea: for f-strings, we add a !d conversion operator, which is superficially similar to !s, !r, and !a. The meaning of !d is: produce the text of the expression (not its value!), followed by an equal sign, followed by the repr of the value of the expression. So: value = 10 s = 'a string!' print(f'{value!d}') print(f'next: {value+1!d}') print(f'{s!d}') produces: value=10 next: value+1=11 s='a string!' I’m not proposing this for str.format(). It would only really make sense for named arguments, and I don’t think print('{value!d}'.format(value=value) is much of a win. The result is a string, so if you really wanted to, you could use a string formatting spec. So: print(f'*{value!d:^20}*' would produce: * value=10 * Although I don’t think that would be very useful in general. The mnemonic is !d for “debugging”. I’d wanted to use !=, because there’s an equal sign involved in the result, but = is the one character that can’t be used after ! (it’s “not equal” in expressions, and f-strings look specifically for that case). I also mentioned !!, but I think I prefer !d as being less confusing. This would be used in debugging print statements, that currently end up looking like: print(f'value={value!r}') and would now be: print(f'{value!d}') There have been discussions about ways to specify str() vs. repr(), using characters other than '=', adding spaces, etc. But they all end up over-complicating what should be a simple tool, not a Swiss Army knife. Thoughts? Eric

12 36

Proposal: "?" Documentation Operator and easy reference to argument types/defaults/docstrings
by Peter O'Connor 29 Apr '19

29 Apr '19

Dear all, Despite the general beauty of Python, I find myself constantly violating the "don't repeat yourself" maxim when trying to write clear, fully documented code. Take the following example: def func_1(a: int = 1, b: float = 2.5) -> float: """ Something about func_1 :param a: Something about param a :param b: Something else about param b :return: Something about return value of func_1 """ return a*b def func_2(c:float=3.4, d: bool =True) -> float: """ Something about func_2 :param c: Something about param c :param d: Something else about param d :return: Something about return value """ return c if d else -c def main_function(a: int = 1, b: float = 2.5, d: bool = True) -> float: """ Something about main_function :param a: Something about param a :param b: Something else about param b :param d: Something else about param d :return: Something about return value """ return func_2(func_1(a=a, b=b), d=d) Which has the following problems: - Defaults are defined in multiple places, which very easily leads to bugs (I'm aware of **kwargs but it obfuscates function interfaces and usually does more harm than good) - Types are defined in multiple places - Documentation is copy-pasted when referencing a single thing from different places. (I can't count the number of types I've written ":param img: A (size_y, size_x, 3) RGB image" - I could now just reference a single RGB_IMAGE_DOC variable) - Argument names need to be written twice - in the header and documentation - and it's up to the user / IDE to make sure they stay in sync. I propose to resolve this with the following changes: - Argument/return documentation can be made inline with a new "?" operator. Documentation becomes a first class citizen. - Argument (type/default/doc) can be referenced by "func.args.<arg_name>.type" / "func.args.<arg_name>.default" / "func.args.<arg_name>.doc". Positional reference: e.g. "func.args[1].default" also allowed. If not specified, they take a special, built-in "Undefined" value (because None may have another meaning for defaults). Return type/doc can be referenced with "func.return.type" / "func.return.doc". This would result in the following syntax: def func_1( a: int = 1 ? 'Something about param a', b: float = 2.5 ? 'Something else about param b', ) -> float ? 'Something about return value of func_1': """ Something about func_1 """ return a*b def func_2( c: float=3.4 ? 'Something about param c', d: bool =True ? 'Something else about param d', ) -> float ? 'Something about return value': """ Something about func_2 """ return c if d else -c def main_function( a: func_1.args.a.type = func_1.args.a.default ? func_1.args.a.doc, b: func_1.args.b.type = func_1.args.b.default ? func_1.args.b.doc, d: func_2.args.d.type = func_2.args.d.default ? func_2.args.d.doc, ) -> func_2.return.type ? func2.return.doc: """ Something about main_function """ return func_2(func_1(a=a, b=b), d=d) If the main_function header seems repetitious (it does) we could allow for an optional shorthand notation like: def main_function( a :=? func_1.args.a, b :=? func_1.args.b, d :=? func_2.args.d, ) ->? func_2.return: """ Something about main_function """ return func_2(func_1(a=a, b=b), d=d) Where "a :=? func_1.args.a" means "argument 'a' takes the same type/default/documentation as argument 'a' of func_1". So what do you say? Yes it's a bold move, but I think in the long term it's badly needed. Perhaps something similar has been proposed already that I'm not aware of.

6 7

Users of statistics software, what quantile functionality would be useful for you?
by Steven D'Aprano 28 Apr '19

28 Apr '19

The statistics module is soon to get a quantile function. For those of you who use statistics software (whether in Python, or using other languages) and specifically use quantiles, what sort of functionality would be useful to you? For example: - evenly-spaced quantiles (say, at 0.25, 0.5, 0.75)? - unevenly-spaced quantiles (0.25, 0.8, 0.9, 0.995)? - one quantile at a time? - any specific definition? - quantiles of a distribution? - anything else? Thanks in advance. -- Steven

4 3

Idea: Allow multiple levels of tracers
by Ram Rachum 28 Apr '19

28 Apr '19

Hi, Here's something I want in Python: Multiple levels of tracers working on top of each other, instead of just one. I'm talking about the tracer that one can set by calling sys.settrace. I've recently released PySnooper: https://github.com/cool-RR/PySnooper/ One of the difficulties I have, is that I can't debug or run the `coverage` tool on the core of this module. That's because the core is a trace function, and debuggers and coverage tools work by setting a trace function. When PySnooper sets its trace function using `sys.settrace`, the code that runs in that trace function runs without getting traced by the coverage tracer. This means that people who develop debuggers and coverage tools can't use a debugger or a coverage tool on the core of their tool. It's quite an annoying problem. My proposed solution: Multiple levels of tracing, instead of just one. When you install a tracer, you're not replacing the existing one, you're appending a tracer to the existing list of tracers. If this was implemented, then when PySnooper would install its tracer, the coverage tracer would still be active and running, for every line of code including the ones in PySnooper's tracer. Obviously, we'll need to figure out the API and any other kind of problems with this proposal. What do you think? Thanks, Ram.

5 12

Syntax to conditionally define a field in a dict
by Joshua Marshall 27 Apr '19

27 Apr '19

Hello all, I have a use case where I need to send a `dict` to a module as an argument. Inside of this, it has a multi-level structure, but each field I need to set may only be set to a single value. Fields must be valid, non-empty strings. It looks a lot like the following in my code: ``` def my_func(val_1, val_2): return { "field_1": val_1, "next_depth": { "field_2": val_2 } } ``` What I want to do is: ``` def my_func(val_1, val_2): return { "field_1": val_1 if val_1, "next_depth": { "field_2": val_2 if val_2 } } ``` Or: ``` def my_func(val_1, val_2): return { if val_1 : "field_1": val_1, "next_depth": { if val_2: "field_2": val_2 } } ``` Where each conditional in this context functions as: ``` if value: d["your_key"] = value ``` for each conditionally added and set key. >From the slack channel #learning_python, there are a number of more general points which need to be handled. The more core syntax, which should be valid throughout the language, would be to have statements like `x = y if cond` and `x[y if cond]`. The first of these intuitively reorganizes to `if cond: x = y`, but the second is not as clear, with a likely equivalent of `if cond: x[y] else raise Exception`. Thanks to Tom Forbes and Jim Kelly for helping critique the idea thus far. -- Please be advised that this email may contain confidential information. If you are not the intended recipient, please notify us by email by replying to the sender and delete this message. The sender disclaims that the content of this email constitutes an offer to enter into, or the acceptance of, any agreement; provided that the foregoing does not invalidate the binding effect of any digital or other electronic reproduction of a manual signature that is included in any attachment. <https://twitter.com/arroyo_networks> <https://www.linkedin.com/company/arroyo-networks> <https://www.github.com/ArroyoNetworks>

7 8

What are the strong use cases for str.rindex()?
by 林自均 26 Apr '19

26 Apr '19

Hi all, I found that there are str.index() and str.rindex(), but there is only list.index() and no list.rindex(). So I filed the issue https://bugs.python.org/issue36639 to provide list.rindex(). However, the issue was rejected and closed with the comment: > There were known, strong use cases for str.rindex(). The list.rindex() method was intentionally omitted. AFAICT no compelling use cases have arisen, so we should continue to leave it out. In general, we don't grow the core APIs unnecessarily. However, I am not sure what the known, strong use cases for str.rindex() are. Why doesn't the strong use cases apply on list.rindex()? Could anyone give me some examples? Thanks. Best, John Lin

7 11