Mailman 3 March 2019 - Python-ideas

Start argument for itertools.accumulate() [Was: Proposal: A Reduce-Map Comprehension and a "last" builtin]
by Raymond Hettinger 23 Oct '19

23 Oct '19

> On Friday, April 6, 2018 at 8:14:30 AM UTC-7, Guido van Rossum wrote: > On Fri, Apr 6, 2018 at 7:47 AM, Peter O'Connor <peter.ed...(a)gmail.com> wrote: >> So some more humble proposals would be: >> >> 1) An initializer to itertools.accumulate >> functools.reduce already has an initializer, I can't see any controversy to adding an initializer to itertools.accumulate > > See if that's accepted in the bug tracker. It did come-up once but was closed for a number reasons including lack of use cases. However, Peter's signal processing example does sound interesting, so we could re-open the discussion. For those who want to think through the pluses and minuses, I've put together a Q&A as food for thought (see below). Everybody's design instincts are different -- I'm curious what you all think think about the proposal. Raymond --------------------------------------------- Q. Can it be done? A. Yes, it wouldn't be hard. _sentinel = object() def accumulate(iterable, func=operator.add, start=_sentinel): it = iter(iterable) if start is _sentinel: try: total = next(it) except StopIteration: return else: total = start yield total for element in it: total = func(total, element) yield total Q. Do other languages do it? A. Numpy, no. R, no. APL, no. Mathematica, no. Haskell, yes. * http://docs.scipy.org/doc/numpy/reference/generated/numpy.ufunc.accumulate.… * https://stat.ethz.ch/R-manual/R-devel/library/base/html/cumsum.html * http://microapl.com/apl/apl_concepts_chapter5.html \+ 1 2 3 4 5 1 3 6 10 15 * https://reference.wolfram.com/language/ref/Accumulate.html * https://www.haskell.org/hoogle/?hoogle=mapAccumL Q. How much work for a person to do it currently? A. Almost zero effort to write a simple helper function: myaccum = lambda it, func, start: accumulate(chain([start], it), func) Q. How common is the need? A. Rare. Q. Which would be better, a simple for-loop or a customized itertool? A. The itertool is shorter but more opaque (especially with respect to the argument order for the function call): result = [start] for x in iterable: y = func(result[-1], x) result.append(y) versus: result = list(accumulate(iterable, func, start=start)) Q. How readable is the proposed code? A. Look at the following code and ask yourself what it does: accumulate(range(4, 6), operator.mul, start=6) Now test your understanding: How many values are emitted? What is the first value emitted? Are the two sixes related? What is this code trying to accomplish? Q. Are there potential surprises or oddities? A. Is it readily apparent which of assertions will succeed? a1 = sum(range(10)) a2 = sum(range(10), 0) assert a1 == a2 a3 = functools.reduce(operator.add, range(10)) a4 = functools.reduce(operator.add, range(10), 0) assert a3 == a4 a4 = list(accumulate(range(10), operator.add)) a5 = list(accumulate(range(10), operator.add, start=0)) assert a5 == a6 Q. What did the Python 3.0 Whatsnew document have to say about reduce()? A. "Removed reduce(). Use functools.reduce() if you really need it; however, 99 percent of the time an explicit for loop is more readable." Q. What would this look like in real code? A. We have almost no real-world examples, but here is one from a StackExchange post: def wsieve(): # wheel-sieve, by Will Ness. ideone.com/mqO25A->0hIE89 wh11 = [ 2,4,2,4,6,2,6,4,2,4,6,6, 2,6,4,2,6,4,6,8,4,2,4,2, 4,8,6,4,6,2,4,6,2,6,6,4, 2,4,6,2,6,4,2,4,2,10,2,10] cs = accumulate(cycle(wh11), start=11) yield( next( cs)) # cf. ideone.com/WFv4f ps = wsieve() # codereview.stackexchange.com/q/92365/9064 p = next(ps) # 11 psq = p*p # 121 D = dict( zip( accumulate(wh11, start=0), count(0))) # start from sieve = {} for c in cs: if c in sieve: wheel = sieve.pop(c) for m in wheel: if not m in sieve: break sieve[m] = wheel # sieve[143] = wheel@187 elif c < psq: yield c else: # (c==psq) # map (p*) (roll wh from p) = roll (wh*p) from (p*p) x = [p*d for d in wh11] i = D[ (p-11) % 210] wheel = accumulate(cycle(x[i:] + x[:i]), start=psq) p = next(ps) ; psq = p*p next(wheel) ; m = next(wheel) sieve[m] = wheel

17 49

support toml for pyproject support
by Jimmy Girardet 29 Jul '19

29 Jul '19

Hi, I don't know if this was already debated but I don't know how to search in the whole archive of the list. For now the adoption of pyproject.toml file is more difficult because toml is not in the standard library. Each tool which wants to use pyproject.toml has to add a toml lib as a conditional or hard dependency. Since toml is now the standard configuration file format, it's strange the python does not support it in the stdlib lije it would have been strange to not have the configparser module. I know it's complicated to add more and more thing to the stdlib but I really think it is necessary for python packaging being more consistent. Maybe we could thought to a readonly lib to limit the added code. If it's conceivable, I'd be happy to help in it. Nice Day guys and girls. Jimmy

13 15

Re: [Python-ideas] Trigonometry in degrees
by Hugh Fisher 08 May '19

08 May '19

> Date: Thu, 7 Jun 2018 12:33:29 +0000 > From: Robert Vanden Eynde <robertvandeneynde(a)hotmail.com> > To: python-ideas <python-ideas(a)python.org> > Subject: [Python-ideas] Trigonometry in degrees > Message-ID: > > > I suggest adding degrees version of the trigonometric functions in the math module. > > - Useful in Teaching and replacing calculators by python, importing something is seen by the young students much more easy than to define a function. I agree that degrees are useful for teaching. They are also very useful for graphics programming, especially with my favourite OpenGL API. But I think that the use of radians in programming language APIs is more prevalent, so the initial advantage of easy learning will be outweighed by the long term inconvenience of adjusting to what everyone else is doing. Writing degrees(x) and radians(x) is a little inconvenient, but it does make it clear what units are being used. And even if your proposal is adopted, there is still going to be a lot of code around that uses the older math routines. With the current API it is a least safe to assume that angles are radians unless stated otherwise. > - Special values could be treated, aka when the angle is a multiple of 90, young students are often surprise to see that cos(pi/2) != 0 > > Testing for a special value Isn't very costly (x % 90 == 0) but it could be pointed out that there is a small overhead using the "degrees" equivalent of trig function because of the radians to degrees conversion And the special values testing. Not just young students :-) I agree with this, but I would prefer the check to be in the implementation of the existing functions as well. Any sin/cos very close to 0 becomes 0, any close to 1 becomes 1. > - Standard names will be chosen so that everyone will use the same name convention. I suggest adding a "d" like sind, cosd, tand, acosd, asind, atand, atan2d. Not "d". In the OpenGL 3D API, and many 3D languages/APIs since, appending "d" means "double precision". It's even sort of implied by the C math library which has sinf and friends for single precision. > > Creating a new package like 'from math.degrees import cos' however I would not recommend that because "cos" in the source code would mean to lookup the import to know if it's in degrees or radians (and that leads to very filthy bugs). Also "degrees" is already so the name would have to change the name of the package. Agree, not a good idea. -- cheers, Hugh Fisher

16 29

f-string "debug" conversion
by Eric V. Smith 03 May '19

03 May '19

This idea was proposed to me at the core sprints last month by Larry Hastings. I've discussed it with a few people, who seem generally positive about it, and we've tweaked it a little bit. I've spent some time implementing it, and I think it's doable. I thought I'd post it here for any additional feedback. Here’s the idea: for f-strings, we add a !d conversion operator, which is superficially similar to !s, !r, and !a. The meaning of !d is: produce the text of the expression (not its value!), followed by an equal sign, followed by the repr of the value of the expression. So: value = 10 s = 'a string!' print(f'{value!d}') print(f'next: {value+1!d}') print(f'{s!d}') produces: value=10 next: value+1=11 s='a string!' I’m not proposing this for str.format(). It would only really make sense for named arguments, and I don’t think print('{value!d}'.format(value=value) is much of a win. The result is a string, so if you really wanted to, you could use a string formatting spec. So: print(f'*{value!d:^20}*' would produce: * value=10 * Although I don’t think that would be very useful in general. The mnemonic is !d for “debugging”. I’d wanted to use !=, because there’s an equal sign involved in the result, but = is the one character that can’t be used after ! (it’s “not equal” in expressions, and f-strings look specifically for that case). I also mentioned !!, but I think I prefer !d as being less confusing. This would be used in debugging print statements, that currently end up looking like: print(f'value={value!r}') and would now be: print(f'{value!d}') There have been discussions about ways to specify str() vs. repr(), using characters other than '=', adding spaces, etc. But they all end up over-complicating what should be a simple tool, not a Swiss Army knife. Thoughts? Eric

12 36

Built-in parsing library
by Nam Nguyen 10 Apr '19

10 Apr '19

Hello list, What do you think of a universal parsing library in the stdlib mainly for use by other libraries in the stdlib? Through out the years we have had many issues with protocol parsing. Some have even introduced security bugs. The main cause of these issues is the use of simple regular expressions. Having a universal parsing library in the stdlib would help cut down these issues. Such a library should be minimal yet encompassing, and whole parse trees should be entirely expressible in code. I am thinking of combinatoric parsing as the main candidate that fits this bill. What do you say? Thanks! Nam

12 20

Re: [Python-ideas] New Project to Capture summaries from this
by Richard Whitehead 30 Mar '19

30 Mar '19

Chris, As a new member to this list, I can tell you that searching for relevant old content was effectively impossible, so I'm all for some way of doing that. Please can I make a more radical suggestion, though: Drop the mailing list. How about a GitHub repo - a specific one (with no code), specifically for early ideas? Then, if an idea was accepted and turned into an issue to be implemented, it could link back to that original discussion. GitHub is easily searchable. It can email you if someone comments on an issue you have raised, etc. An alternative might be a StackOverflow section, but that wouldn't provide such tight integration in the case of an issue being raised. The new work you're doing would be a good way to populate the repo with its initial content. Richard

10 15

Unified style of cache management API
by Ma Lin 30 Mar '19

30 Mar '19

re module [1] and struct module [2] have module-level cache for compiled stuffs. Other third-party modules may also need cache for something. Do we need an unified cache management API like this? I suppose it's not mandatory, but welcome each module to use this API. module.cache_get_capacity() # return current capacity module.cache_set_capacity(100) # set capacity module.cache_clear() # clear cache Moreover, add these API to sys module, then the users can manage system wide cache easily: sys.cache_register(f) # register a .cache_clear() function sys.cache_clear() # call all registered .cache_clear() [1] re module policy: FIFO capacity: 512 (default), changeable implementation: Python https://github.com/python/cpython/blob/v3.8.0a3/Lib/re.py#L268-L295 [2] struct module clear entire cache when full capacity: 100, unchangeable implementation: C https://github.com/python/cpython/blob/v3.8.0a3/Modules/_struct.c#L2071-L21…

3 6

Re: [Python-ideas] New Project to Capture summaries from this
by Paul Moore 29 Mar '19

29 Mar '19

Bah. Tablet client defaults to reply, not reply to all... On Fri, 29 Mar 2019 at 08:10, Paul Moore <p.f.moore(a)gmail.com> wrote: > On Fri, 29 Mar 2019 at 03:25, Chris Angelico <rosuav(a)gmail.com> wrote: > >> On Fri, Mar 29, 2019 at 2:07 PM Stephen J. Turnbull >> <turnbull.stephen.fw(a)u.tsukuba.ac.jp> wrote: >> > >> > There's a big difference, though. Email users choose their own email >> > clients. If you choose GMail, well, "sorry, you chose GMail." (I >> > understand that avoiding GMail on Apple handhelds is kinda hard, >> > AppleMail sucking amazingly and all. But writing technical discussion >> > on a phone is strictly for disasters, anyway, at least IME.) >> >> That's half of my point (the distinction between "suboptimal clients" >> and "suboptimal services"), but the other half is that every time >> someone says "sorry, you chose Gmail", there's a lengthy discussion >> that ends up NOT showcasing any sort of perfect alternative - and >> often not even any *better* alternatives. Have you ever actually >> convinced someone to move off Gmail onto some other client? > > > As someone who uses gmail (the web interface) this is a good point. When > people say that there are all sorts of better alternative clients, no-one > has ever been able to offer one that actually satisfies my specific > requirements. Having said that, *in spite of having to use gmail* I still > strongly prefer mailing lists. > > I can’t easily articulate why, but certainly one aspect of it is the fact > that there is no universally accepted alternative that gets proposed. One > time it’s discourse, then it’s github, then something else I’ve never heard > of... And every other project that gets quoted as having “successfully > switched” seems to use something different. So in my mind the question > isn’t about mail or a particular alternative (that I can look at and form > an opinion about over time, as these discussions reoccur) but rather about > mail or “not mail” with an ever changing alternative that I have to > consider and re-assess from scratch each time. > > For me, mail wins as the stable alternative. And for something I spend so > much time on, stability is essential. > > Paul >

1 0

The Mailing List Digest Project
by Abdur-Rahmaan Janhangeer 28 Mar '19

28 Mar '19

As proposed on python-ideas, i setup a repo to turn mail threads into articles. here is the repo https://github.com/Abdur-rahmaanJ/py-mailing-list-summary i included a script to build .md to .html (with syntax highlighting) here is the index https://abdur-rahmaanj.github.io/py-mailing-list-summary/ included 3 articles as a start if you want to contribute an article, just follow existing .md format and put it in the .md folder planning to go across ideas, list and dev i can tell you, it's a really enjoyable experience. psst. we can enhance some html later -- Abdur-Rahmaan Janhangeer Mauritius

2 5

New Project to Capture summaries from this list
by Christopher Barker 28 Mar '19

28 Mar '19

Hi all, Inspired by Abdur-Rahmaan Janhangeer and some previous discussion son this list, I've started a project to capture some of the common threads from this list (and others). We often find ourselves saying things like "this was discussed on this list a year or so ago" when someone brings up a common topic, but it can be hard to find those threads, and often those threads are very, very, long and hard to follow. I'm hoping you all will contribute to this effort to better document the history of Python-ideas. The project is here: https://github.com/PythonCHB/PythonListsSummaries And is published here: https://pythonchb.github.io/PythonListsSummaries If it catches on, we can move it to maybe a new gitHub org, or somewhere less associated with me -- I'm hoping it will be a community effort. Contributions Encouraged! I've kicked it off with a summary of the whole ``list.join()`` question: https://pythonchb.github.io/PythonListsSummaries/python_ideas/join_as_a_lis… -Chris -- Christopher Barker, PhD Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython

1 0