Mailman 3 November 2018 - Python-ideas

Re: [Python-ideas] Trigonometry in degrees
by Hugh Fisher 08 May '19

08 May '19

> Date: Thu, 7 Jun 2018 12:33:29 +0000 > From: Robert Vanden Eynde <robertvandeneynde(a)hotmail.com> > To: python-ideas <python-ideas(a)python.org> > Subject: [Python-ideas] Trigonometry in degrees > Message-ID: > > > I suggest adding degrees version of the trigonometric functions in the math module. > > - Useful in Teaching and replacing calculators by python, importing something is seen by the young students much more easy than to define a function. I agree that degrees are useful for teaching. They are also very useful for graphics programming, especially with my favourite OpenGL API. But I think that the use of radians in programming language APIs is more prevalent, so the initial advantage of easy learning will be outweighed by the long term inconvenience of adjusting to what everyone else is doing. Writing degrees(x) and radians(x) is a little inconvenient, but it does make it clear what units are being used. And even if your proposal is adopted, there is still going to be a lot of code around that uses the older math routines. With the current API it is a least safe to assume that angles are radians unless stated otherwise. > - Special values could be treated, aka when the angle is a multiple of 90, young students are often surprise to see that cos(pi/2) != 0 > > Testing for a special value Isn't very costly (x % 90 == 0) but it could be pointed out that there is a small overhead using the "degrees" equivalent of trig function because of the radians to degrees conversion And the special values testing. Not just young students :-) I agree with this, but I would prefer the check to be in the implementation of the existing functions as well. Any sin/cos very close to 0 becomes 0, any close to 1 becomes 1. > - Standard names will be chosen so that everyone will use the same name convention. I suggest adding a "d" like sind, cosd, tand, acosd, asind, atand, atan2d. Not "d". In the OpenGL 3D API, and many 3D languages/APIs since, appending "d" means "double precision". It's even sort of implied by the C math library which has sinf and friends for single precision. > > Creating a new package like 'from math.degrees import cos' however I would not recommend that because "cos" in the source code would mean to lookup the import to know if it's in degrees or radians (and that leads to very filthy bugs). Also "degrees" is already so the name would have to change the name of the package. Agree, not a good idea. -- cheers, Hugh Fisher

16 29

f-string "debug" conversion
by Eric V. Smith 03 May '19

03 May '19

This idea was proposed to me at the core sprints last month by Larry Hastings. I've discussed it with a few people, who seem generally positive about it, and we've tweaked it a little bit. I've spent some time implementing it, and I think it's doable. I thought I'd post it here for any additional feedback. Here’s the idea: for f-strings, we add a !d conversion operator, which is superficially similar to !s, !r, and !a. The meaning of !d is: produce the text of the expression (not its value!), followed by an equal sign, followed by the repr of the value of the expression. So: value = 10 s = 'a string!' print(f'{value!d}') print(f'next: {value+1!d}') print(f'{s!d}') produces: value=10 next: value+1=11 s='a string!' I’m not proposing this for str.format(). It would only really make sense for named arguments, and I don’t think print('{value!d}'.format(value=value) is much of a win. The result is a string, so if you really wanted to, you could use a string formatting spec. So: print(f'*{value!d:^20}*' would produce: * value=10 * Although I don’t think that would be very useful in general. The mnemonic is !d for “debugging”. I’d wanted to use !=, because there’s an equal sign involved in the result, but = is the one character that can’t be used after ! (it’s “not equal” in expressions, and f-strings look specifically for that case). I also mentioned !!, but I think I prefer !d as being less confusing. This would be used in debugging print statements, that currently end up looking like: print(f'value={value!r}') and would now be: print(f'{value!d}') There have been discussions about ways to specify str() vs. repr(), using characters other than '=', adding spaces, etc. But they all end up over-complicating what should be a simple tool, not a Swiss Army knife. Thoughts? Eric

12 36

Return for assignment blocks
by Calvin Spealman 20 Mar '19

20 Mar '19

I'd like to suggest what I think would be a simple addition to `def` and `class` blocks. I don't know if calling those "Assignment Blocks" is accurate, but I just mean to refer to block syntaxes that assign to a name. Anyway, I propose a combined return-def structure, and optionally also allowing a return-class version. Omitting the name would be allowable, as well. This would only apply to a `def` or `class` statement made as the last part of the function body, of course. def ignore_exc(exc_type): return def (func): @wraps(func) return def (*args, **kwargs): try: return func(*args, **kwargs) except exc_type: pass Thanks for considering and for any comments, thoughts, or feedback on the idea!

14 33

Support localization of unicode descriptions
by Pander 25 Feb '19

25 Feb '19

Dear all, Who might also be interested in setting up a project that supports localization for Unicode block description and character description. Translations are available from https://github.com/unicode-table/unicode-table-data/tree/master/loc <https://github.com/unicode-table/unicode-table-data/tree/master/loc> If possible, use a gettext approach similar to https://pypi.org/project/pycountry/ <https://pypi.org/project/pycountry/> Implementing this feature will allow users to read Unicode descriptions in their own language, other than English. For example, now is possible only in English: from unicodedata import name print(name('ß')) LATIN SMALL LETTER SHARP S So unicodedata could provide a way to translate LATIN SMALL LETTER SHARP S to e.g. German with: from unicodedata import name from unicodedata_l10n import LOCALED_DIR from gettext import translation german = translation('UnicodeData' LOCALED_DIR, languages=['de']) german.install() print(_(name('ß'))) LATEINISCHER KLEINBUCHSTABE SCHARFES S and something similar for unicodedata.category Best, Pander

5 8

Dart like multi line strings identation
by Marius Räsener 08 Feb '19

08 Feb '19

Hey List, this is my very first approach to suggest a Python improvement I'd think worth discussing. At some point, maybe with Dart 2.0 or a little earlier, Dart is now supporting multiline strings with "proper" identation (tried, but I can't find the according docs at the moment. probably due to the rather large changes related to dart 2.0 and outdated docs.) What I have in mind is probably best described with an Example: print(""" I am a multiline String. """) the closing quote defines the "margin indentation" - so in this example all lines would get reduces by their leading 4 spaces, resulting in a "clean" and unintended string. anyways, if dart or not, doesn't matter - I like the Idea and I think python3.x could benefit from it. If that's possible at all :) I could also imagine that this "indentation cleanup" only is applied if the last quotes are on their own line? Might be too complicated though, I can't estimated or understand this... thx for reading, Marius

23 47

__len__() for map()
by Kale Kundert 13 Dec '18

13 Dec '18

I just ran into the following behavior, and found it surprising: >>> len(map(float, [1,2,3])) TypeError: object of type 'map' has no len() I understand that map() could be given an infinite sequence and therefore might not always have a length. But in this case, it seems like map() should've known that its length was 3. I also understand that I can just call list() on the whole thing and get a list, but the nice thing about map() is that it doesn't copy data, so it's unfortunate to lose that advantage for no particular reason. My proposal is to delegate map.__len__() to the underlying iterable. Similarly, map.__getitem__() could be implemented if the underlying iterable supports item access: class map: def __init__(self, func, iterable): self.func = func self.iterable = iterable def __iter__(self): yield from (self.func(x) for x in self.iterable) def __len__(self): return len(self.iterable) def __getitem__(self, key): return self.func(self.iterable[key]) Let me know if there any downsides to this that I'm not seeing. From my perspective, it seems like there would be only a number of (small) advantages: - Less surprising - Avoid some unnecessary copies - Backwards compatible -Kale

19 113

[Brainstorm] Testing with Documented ABCs
by Abe Dillon 08 Dec '18

08 Dec '18

I've been pulling a lot of ideas from the recent discussion on design by contract (DBC), the elegance and drawbacks <https://bemusement.org/doctests-arent-code> of doctests <https://docs.python.org/3/library/doctest.html>, and the amazing talk <https://www.youtube.com/watch?v=MYucYon2-lk> given by Hillel Wayne at this year's PyCon entitled "Beyond Unit Tests: Taking your Tests to the Next Level". To recap a lot of previous discussions: - Documentation should tell you: A) What a variable represents B) What kind of thing a variable is C) The acceptable values a variable can take - Typing and Tests can partially take the place of documentation by filling in B and C (respectively) and sometimes A can be inferred from decent naming and context. - Contracts can take the place of many tests (especially when combined with a library like hypothesis) - Contracts/assertions can provide "stable" documentation in the sense that it can't get out of sync with the code. - Attempts to implement contracts using standard Python syntax are verbose and noisy because they rely heavily on decorators that add a lot of repetitive preamble to the methods being decorated. They may also require a metaclass which restricts their use to code that doesn't already use a metaclass. - There was some discussion about the importance of "what a variable represents" which pointed to this article <http://pgbovine.net/python-unreadable.htm> by Philip J. Guo (author of the magnificent pythontutor.com). I believe Guo's usage of "in-the-small" and "in-the-large" are confusing because a well decoupled program shouldn't yield functions that know or care how they're being used in the grand machinations of your project. The examples he gives are of functions that could use a doc string and some type annotations, but don't actually say how they relate to the rest of the project. One thing that caught me about Hillel Wayne's talk was that some of his examples were close to needing practically no code. He starts with: def tail(lst: List[Any]) -> List[Any]: assert len(lst) > 0, "precondition" result = lst[1:] assert [lst[0]] + result == lst, "postcondition" return result He then re-writes the function using a contracts library: @require("lst must not be empty", lambda args: len(args.lst) > 0) @ensure("result is tail of lst", lambda args, result: [args.lst[0]] + result == args.lst) def tail(lst: List[Any]) -> List[Any]: return lst[1:] He then writes a unit test for the function: @given(lists(integers(), 1)) def test_tail(lst): tail(lst) What strikes me as interesting is that the test pretty-much doesn't need to be written. The 'given' statement should be redundant based on the type annotation and the precondition. Anyone who knows hypothesis, just imagine the @require is a hypothesis 'assume' call. Furthermore, hypothesis should be able to build strategies for more complex objects based on class invariants and attribute types: @invariant("no overdrafts", lambda self: self.balance >= 0) class Account: def __init__(self, number: int, balance: float = 0): super().__init__() self.number: int = number self.balance: float = balance A library like hypothesis should be able to generate valid account objects. Hypothesis also has stateful testing <https://hypothesis.readthedocs.io/en/1.4.1/stateful.html> but I think the implementation could use some work. As it is, you have inherit from a class that uses a metaclass AND you have to pollute your class's name-space with helper objects and methods. If we could figure out a cleaner syntax for defining invariants, preconditions, and postconditions we'd be half-way to automated testing UTOPIA! (ok, maybe I'm being a little over-zealous) I think there are two missing pieces to this testing problem: side-effect verification and failure verification. Failure verification should test that the expected exceptions get thrown when known bad data is passed in or when an object is put in a known illegal state. This should be doable by allowing Hypothesis to probe the bounds of unacceptable input data or states, though it might seem a bit silly because if you've already added a precondition, "x >= 0" to a function, then it obviously should raise a PreconditionViolated when passed any x < 0. It may be important, however; if for performance reasons, you need to disable invariant checking but you still want certain bad input to raise exceptions, or your system has two components that interact with slightly mis-matched invariants and you want to make sure the components handle the edge-condition correctly. You can think of Types from a set-theory perspective where the Integer type is conceptually the set of all integers, and invariants would specify a smaller subset than Typing alone, however if the set of all valid outputs of one component is not completely contained within the set of all valid inputs to another component, then there will be edge-cases resulting from the mismatch. In that sense, some of the invariant verification could be static-ish (as much as Python allows). Side-effect verification is usually done by mocking dependencies. You pass in a mock database connection and make sure my object sends and receives data as expected. As crazy as it sounds, this too can be almost completely automated away if all of the above tools are in place AND if Python gained support for Exception annotations. I wrote a Java (yuck) library at work that does this. I wan't to port it to Python and share it, but it basically enumerates a bunch of stuff: the "sources" and "destinations" of the system, how those relate to dependencies, how they relate to each other (if dependency X is unresponsive, I can't get sources A, B, or G and if I can't get source B, I can't write destination Y), the dependency failure modes (Exceptions raised, timeouts, unrecognized key, missing data, etc.), all the public methods of the class under test and what sources and destinations they use. Then I enumerate 'k' from 0 to some limit for the max number of simultaneous faults to test for: Then for each method that can have n >= k simultaneous faults I test all (n choose k) combinations of faults for that method against the desired behavior. I'm sure that explanation is as clear as mud. I will try to get a working Python example at some point to demonstrate. Finally, in the PyCon video; Hillel Wayne shows an example of testing that an "add" function is commutative. It seems that once you write that invariant, it might apply to many different functions. A similar invariant may be "reversibility" like: @given(text()) def test_reversable_codex(s): assert s == decode(encode(s)), "not reversible" That might be a common property that other functions share: @invariant(reversible(decode)) def encode(s: str) -> bytes: ... Having said all that, I wanted to brainstorm some possible solutions for implementing some or all of the above in Python without drowning you code in decorators. NOTE: Please don't get hung up on specific syntax suggestions! Try to see the forest through the trees! An example syntax could be: #Instead of this @require("lst must not be empty", lambda args: len(args.lst) > 0) @ensure("result is tail of lst", lambda args, result: [args.lst[0]] + result == args.lst) def tail(lst: List[Any]) -> List[Any]: return lst[1:] #Maybe this? non_empty = invariant("Must not be empty", lambda x: len(x) > 0) # can be re-used def tail(lst: List[Any] d"Description of what this param represents. {non_empty}") -> List[Any] d"Description of return value {lst == [lst[0]] + __result__}": """ Description of function """ return lst[1:] Python could build the full doc string like so: """ Description of function Args: lst: Description of what this param represents. Must not be empty. Returns: Description of return value. """ d-strings have some description followed by some terminator after which either invariant objects or [optionally strings] followed by an expression on the arguments and __return__? I'm sorry this is so half-baked. I don't really like the d-string concept and I'm pretty sure there are a million problems with it. I'll try to flesh out the side-effect verification concept more later along with all the other poorly explained stuff. I just wanted to get these thoughts out for discussion, but now it's super late and I have to go!

10 24

[off-topic?] Unwinding generators
by Juancarlo Añez 30 Nov '18

30 Nov '18

This is code from the Twisted library: https://github.com/twisted/twisted/blob/trunk/src/twisted/internet/defer.py… It "unwinds" a generator to yield a result before others. I don't have hard evidence, but my experience is that that kind of manipulation leaks resources, specially if exceptions escape from the final callback. Is there a bug in exception handling in the generator logic, or is unwinding just inherently wrong? How could the needs tried to solved with unwinding be handled with async? -- Juancarlo *Añez*

1 0

Make None a subclass of int [alternative to iNaN]
by Ken Hilton 27 Nov '18

27 Nov '18

Hi all, Reading the iNaN discussion, most of the opposition seems to be that adding iNaN would add a new special value to integers and therefore add new complexity. I propose, instead, that we make None a subclass of int (or even a certain value of int) to represent iNaN. Therefore: >>> None + 1, None - 1, None * 2, None / 2, None // 2 (None, None, None, nan, None) # mathematical operations on NaN return NaN >>> None & 1, None | 1, None ^ 1 # I'm not sure about this one. The following could be plausible: (0, 1, 1) # or this might make more sense, as this *is* NaN we're talking about: (None, None, None) >>> isinstance(None, int) True # the whole point of this idea >>> issubclass(type(None), int) True # no matter whether None *is* an int or just a subclass, this will be true as issubclass(int, int) is True I know this is a crazy idea, but I thought it could have some merit, so why not throw it out here? Sharing, Ken Hilton;

3 2

hybrid regex engine: backtracking + Thompson NFA
by Celelibi 25 Nov '18

25 Nov '18

Hello, I found this topic discussed on the python-dev ML back in 2010 [1]. I'm bringing it up 8 years later with a variation. In short: The article [2] highlight that backtracking-based regex engine (like SRE in python) have pathological cases that run in an exponential time of the input, while they could run in a linear time. Not mentioned by the article is that even quadratic time can be a problem with large inputs. Which happen pretty often when you're looking for delimited stuff like this : re.match(r'.*?,(.*),,', "a,"*10000) Of course, there's a catch. Backreferences most-likely cannot be implemented in a guaranteed linear time, and some cases of look-behind assertions might prove difficult as well. But other features like alternatives (foo|bar) and repetitions (.*) are no problem. The general idea of the Thompson's algorithm is that the simulation of the NFA is basically in all the reachable states at the same time while parsing the input string character by character. Of course, for regex engines that use a VM (like Python's SRE) you can also make the execution of the equivalent byte-code be in several states at once. The 2010 discussion seems to be about having two separate engines and selecting the best one for a given regex. What I'm proposing here is to resort to backtracking only for the regex features that need it. Meaning that within a regex like r'(.*),.* .*,\1' the evaluation of the middle part would use the Thompson's algorithm, while the \1 could trigger the backtracking mechanism if the string doesn't match. What do you think about it? Has this already been discussed and rejected? Is it just a matter of showing the code? (_sre.c seems... non-trivial) Has this already been juged not worth the maintainance effort? AFAIK, there's not many hybrid regex engines out there. But one notable implementation is the third version of the Henry Spencer's regex engine [3]. Which he doesn't seem to have documented publicly, but postgres has done a pretty good job at reverse-engineering a high level view of it [4]. I'm still unsure how the backtracking of some parts of the regex interact with the multi-state evaluation of the other parts. But at least it exists and works, so it is feasible. Best regards, Celelibi [1] https://mail.python.org/pipermail/python-dev/2010-March/098354.html [2] https://swtch.com/~rsc/regexp/regexp1.html [3] https://github.com/garyhouston/hsrex [4] https://github.com/postgres/postgres/tree/master/src/backend/regex

2 1