Mailman 3 September 2016 - Python-ideas

Re: [Python-ideas] Shuffled
by Arek Bulski Sept. 6, 2016

Sept. 6, 2016

shuffled() should be in the random module, of course. I dont suggest a builtin. Although now that you mentioned it, I could go for that too. There are usage cases where its heavily used, in randomized testing for example. I am sure that there are also other domains where randomization of lists is used. Another reason to put it there is that using shuffle is inconvenient. The fact that I CAN write it myself doesnt mean that it doesnt belong in the standard library. Implementing this in pure python wont take a lot of work. pozdrawiam, Arkadiusz Bulski

4 4

Re: [Python-ideas] shuffled
by Arek Bulski Sept. 6, 2016

Sept. 6, 2016

They closed it. Will they accept a patch if I send it? Where is the repo for the code? Never submitted anything to py code base. -- Arkadiusz Bulski --

3 2

Shuffled
by Arek Bulski Sept. 6, 2016

Sept. 6, 2016

We should add shuffled() analog to sorted. Also shuffle() should return self so mutating methods could be chained. -- Arkadiusz Bulski --

4 3

Semantics for type checking (was: What should a good type checker do?)
by Koos Zevenhoven Sept. 4, 2016

Sept. 4, 2016

On Friday, while replying to a post on python-dev about PEP 526 (variable annotations), I ended up mentioning things that I think a good type checker should do, which seem to differ from the general understanding. The discussion should continue here, though, because the aim of PEP 526 is not to define type-checking semantics. I'll recap the thoughts in this post, but here are links to the original posts, for reference: [1] https://mail.python.org/pipermail/python-dev/2016-September/146069.html [2] https://mail.python.org/pipermail/python-dev/2016-September/146090.html Quoting [1], with added explanations in between: On Fri, Sep 2, 2016 at 1:10 PM, Koos Zevenhoven <k7hoven(a)gmail.com> wrote: [...] > > The more you infer types, the less you check them. It's up to the > programmers to choose the amount of annotation. > [...] > > I'm also a little worried about not being able to reannotate a name. > IOW, this should be allowed: x : int = 1 foo(x) x : float = 0.5 bar(x) It turns out that disallowing this was not the PEP's intention, because type-checking semantics is up to the type checker. This has now been clarified in the PEP. Good. [...] >> def eggs(cond:bool): >> x: Optional[List[int]] >> if cond: >> x = None # Now legal >> else: >> x: = [] >> spam(x) >> The above is an example from Mark Shannon's email, and below is my response: > > A good checker should be able to infer that x is a union type at the > point that it's passed to spam, even without the type annotation. For > example: > > def eggs(cond:bool): > if cond: > x = 1 > else: > x = 1.5 > spam(x) # a good type checker infers that x is of type Union[int, float] > Here, there are two inferred types for x, but in different branches of the if statement. So x, which is dynamically typed at runtime, becomes a Union at "static-check time". > Or with annotations: > > def eggs(cond:bool): > if cond: > x : int = foo() # foo may not have a return type hint > else: > x : float = bar() # bar may not have a return type hint > spam(x) # a good type checker infers that x is of type Union[int, float] > > Two explicit annotations, and again neither of them "wins". The inferred type after the if statement is therefore a Union. Below, I'm quoting [2], my response to an email that disagreed with me about the above being something "a good type checker" would do: On Fri, Sep 2, 2016 at 6:49 PM, Koos Zevenhoven <k7hoven(a)gmail.com> wrote: >>> A good checker should be able to infer that x is a union type at the >>> point that it's passed to spam, even without the type annotation. For >>> example: >>> >>> def eggs(cond:bool): >>> if cond: >>> x = 1 >>> else: >>> x = 1.5 >>> spam(x) # a good type checker infers that x is of type Union[int, float] >> >> Oh I really hope not. I wouldn't call that a *good* type checker. I >> would call that a type checker that is overly permissive. > > I guess it's perfectly fine if we disagree about type checking ideals, > and I can imagine the justification for you thinking that way. There > can also be different type checkers, and which can have different > modes. > > But assume (a) that the above function is perfectly working code, and > spam(...) accepts Union[int, float]. Why would I want the type checker > to complain? > > Then again, (b) instead of that being working code, it might be an > error and spam only takes float. No problem, the type checker will > catch that. > > In case of (b), to get the behavior you want (but in my hypothetical > semantics), this could be annotated as > > def eggs(cond:bool): > x : float > if cond: > x = 1 # type checker says error > else: > x = 1.5 > spam(x) > > So here the programmer thinks the type of x should be more constrained > than what spam(...) accepts. The reasoning here is that x is explicitly annotated without an assignment, so the inferred types subsequently assigned to it (int and float) must be compatible with the annotation. Therefore "the good type checker" would say assigning 1 is an error. > > Or you might have something like this > > def eggs(cond:bool): > if cond: > x = 1 > else: > x = 1.5 > # type checker has inferred x to be Union[int, float] > x : float # type checker finds an error > spam(x) > > Here, the same error is found, but at a different location. > Here, x is explicitly annotated as float while its value remains a Union[int, float], as inferred by "the good type checker". Therefore this is an error. One could consider this a type assertation (at static-check time). "The good type checker" should also be able to tell the programmer that it's "x = 1" that made the subsequent "type assertion" fail. In addition to being type *hints*, the annotations can also be considered as "check points". -- Koos -- + Koos Zevenhoven + http://twitter.com/k7hoven +

3 6

Re: [Python-ideas] [Python-Dev] What should a good type checker do? (was: Please reject or postpone PEP 526)
by Stephen J. Turnbull Sept. 4, 2016

Sept. 4, 2016

Please respect Reply-To, set to python-ideas. Greg Ewing writes: > Chris Angelico wrote: > > Forcing people to write 1.0 just to be compatible with 1.5 will cause > > a lot of annoyance. > > Indeed, this would be unacceptable IMO. But "forcing" won't happen. Just ignore the warning. *All* such Python programs will continue to run (or crash) exactly as if the type declarations weren't there. If you don't like the warning, either don't run the typechecker, or change your code to placate it. But allowing escapes from a typechecker means allowing escapes. All of them, not just the ones you or I have preapproved. I want my typechecker to be paranoid, and loud about it. That doesn't mean I would never use a type like "Floatable" (ie, any type subject to implicit conversion to float). But in the original example, I would probably placate the typechecker. YMMV, of course.

3 4

What should a good typechecker do?
by Stephen J. Turnbull Sept. 3, 2016

Sept. 3, 2016

Guido van Rossum writes: > But that's not what type comments mean! They don't annotate the > expression. They annotate the variable. In PEP 484. But syntactically, AFAICS in an initialization that's a distinction without a difference. It would be perfectly possible to write a checker that allows if cond: x: str = "a string, what else?" else: x: int = 1 and infers the union, and even infers the types from the expressions in if cond: x = "a string, what else?" else: x = 1 Once you have that, then the only things that are really being typed are the initializer expressions. I don't understand why some people seem to think that is a *good* typechecker (if that's it's normal mode of operation). But as a tool to identify the types that untyped (or ambiguously-typed) code actually uses, review them, and help produce a precisely typed version, it seems perfectly plausible to me.

2 1

Extending expressions using ellipsis
by Shane Hathaway Sept. 1, 2016

Sept. 1, 2016

Hi, I write a lot of SQLAlchemy code that looks more or less like this: rows = ( dbsession.query(Table1) .join( Table2, Table2.y = Table1.y) .filter(Table1.x = xx) .all()) The expressions get very long and nearly always need to be spread to multiple lines. I've tried various styles and have chosen the style above as the most tasteful available. Pros of the existing syntax: - It's possible to indent clearly and consistently. - Nested indentation works out OK. - It's flexible; I can combine lines or separate them for emphasis. Cons: - Extra parentheses are required. - The indentation is not enforced by the parser, so I have unnecessary freedom that could let various mistakes slip through. - The closing parenthesis has to move every time I append to or reorder the expression, leading to diff noise in version control. (Alternatively, I could put the closing parenthesis on its own line, but that consumes precious vertical reading space.) I'd like to suggest a small change to the Python parser that would make long expressions read better: rows = dbsession.query(Table1) ... .join( Table2, Table2.y = Table1.y) .filter(Table1.x = xx) .all() The idea is to use an ellipsis at the end of a line to spread an expression over multiple indented lines, terminated by a return to an earlier indentation level. You can still indent more deeply as needed, as shown above by the join() method call. This syntax has all the pros of the existing syntax and resolves all the cons: - No extra parentheses are required. - The indentation is enforced, so my mistakes are more likely to be caught early. - Without a closing parenthesis, there is no diff noise when I append to or reorder an expression. I've thought about using a colon instead of an ellipsis, but in Python, a colon starts a list of statements; that's not my goal. Instead, I'm looking for ways to use parser-enforced indentation to avoid mistakes and help my code read better without changing any semantics. Feedback is welcome! Shane

12 18

Improve the error message for impossible multiple inheritance
by Neil Girdhar Sept. 1, 2016

Sept. 1, 2016

I sometimes get the error message: TypeError: Cannot create a consistent method resolution order (MRO) for bases … I suggest that Python report * cycles (if there are any), or * a list of orderings of direct base classes (if there are no cycles). A cycle for the base classes A, B, C, D can be reported as follows: TypeError: Cannot create a consistent method resolution order (MRO) because A inherits from B (through X, Y), but B inherits from A (through Z). The topological sort could be reported as follows for base classes A, B, C, D: TypeError: Cannot create a consistent method resolution order (MRO) because B inherit from C (through X, Y), but B follows C in inheritance order. It might be nice to have an exposed tool somewhere in the standard library that spits out for A, B, C, D: A, B inherit from C (through X, Y) C inherit from D (through Z) Best, Neil

1 0

real numbers with SI scale factors: next steps
by Ken Kundert Sept. 1, 2016

Sept. 1, 2016

Okay, let's try to wrap this up. In summary I proposed three things: 1. A change to the Python lexer to accept SI literal as an alternative, but not replacement to, E-notation. As an optional feature, simple units could be added to the end but would be largely ignored. So the following would be accepted: freq = 2.4GHz r = 1k l = 10nm The idea in accepting units was to allow them to be specified when convenient as additional documentation on the meaning of the number. Objections: a. Acceptance of the abbreviation for Exa (E) overlaps with E-notation (1E+1 could represent 1e18 + 1 or 10). A suggestion to change the prefix from E to X conflicts with a proposal to use X, W, and V to represent 10^27, 10^30, and 10^33 (en.wikipedia.org/wiki/Metric_prefix) b. Allowing the units to be specified will lead some users to assume a dimensional analysis is being performed when in fact the units are ignored. This false sense of security could lead to bugs. c. The proposal only supports simple units, not compound units such as m/s. So even if hooks were provided to allow access to the units to support an add-on dimensional analysis capability, an additional mechanism would have to be provided to support compound units. d. Many people objected to allowing the use of naked scale factors as a perversion of the standard. 2. A change to the float() function so that it accepts SI scale factors and units. This extension naturally follows from the first: the float function should accept anything the Python parser accepts. For example: freq = float('2.4GHz') r = float('1k') l = float('10nm') Objections: a. The Exa objection from the above proposal is problematic here as well. b. Things that used to be errors are now no longer errors. This could cause problems if a program was counting on float('1k') to be an error. 3. A change to the various string formatting mechanisms to allow outputting real numbers with SI scale factors: >>> print('Speed of light in a vacuum: {:r}m/s.'.format(2.9979e+08)) Speed of light in a vacuum: 299.79 Mm/s. >>> print('Speed of sound in water: %rm/s.' % 1481 Speed of sound in water: 1.481 km/s. Objections: No objections were raised that I recall, however here is something else to consider: a. Should we also provide mechanism for the binary scale factors (Ki, Mi, ..., Yi)? For example: '{:b}B'.format(2**30) --> 1 GiB. On proposed extension 1 (native support for SI literals) my conclusion is that we did not reach any sense of consensus and there was considerable opposition to my proposal. There was much less discussion on extensions 2 & 3, so it is hard to say whether consensus was reached. So, given all this, I would like to make the following recommendations: 1. No action should be taken. 2. The main justification to modifying float() was to make it consistent with the extended Python language. Without extension 1, this justification goes away. However the need to be able to easily convert strings of numbers with SI scale factors into floats still exists. This should be handled by adding a library or extending an existing library. 3. Allowing numbers to be formatted with SI prefixes is useful and not controversial. The 'r' and 'b' format codes should be added to the various string formatting mechanisms. What do you think? -Ken

14 28