Mailman 3 September 2016 - Python-ideas

@classproperty, @abc.abstractclasspropery, etc.
by K. Richard Pixley Dec. 16, 2020

Dec. 16, 2020

There's a whole matrix of these and I'm wondering why the matrix is currently sparse rather than implementing them all. Or rather, why we can't stack them as: class foo(object): @classmethod @property def bar(cls, ...): ... Essentially the permutation are, I think: {'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable attribute}. concreteness implicit first arg type name comments {unadorned} {unadorned} method def foo(): exists … [View More]now {unadorned} {unadorned} property @property exists now {unadorned} {unadorned} non-callable attribute x = 2 exists now {unadorned} static method @staticmethod exists now {unadorned} static property @staticproperty proposing {unadorned} static non-callable attribute {degenerate case - variables don't have arguments} unnecessary {unadorned} class method @classmethod exists now {unadorned} class property @classproperty or @classmethod;@property proposing {unadorned} class non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract {unadorned} method @abc.abstractmethod exists now abc.abstract {unadorned} property @abc.abstractproperty exists now abc.abstract {unadorned} non-callable attribute @abc.abstractattribute or @abc.abstract;@attribute proposing abc.abstract static method @abc.abstractstaticmethod exists now abc.abstract static property @abc.staticproperty proposing abc.abstract static non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract class method @abc.abstractclassmethod exists now abc.abstract class property @abc.abstractclassproperty proposing abc.abstract class non-callable attribute {degenerate case - variables don't have arguments} unnecessary I think the meanings of the new ones are pretty straightforward, but in case they are not... @staticproperty - like @property only without an implicit first argument. Allows the property to be called directly from the class without requiring a throw-away instance. @classproperty - like @property, only the implicit first argument to the method is the class. Allows the property to be called directly from the class without requiring a throw-away instance. @abc.abstractattribute - a simple, non-callable variable that must be overridden in subclasses @abc.abstractstaticproperty - like @abc.abstractproperty only for @staticproperty @abc.abstractclassproperty - like @abc.abstractproperty only for @classproperty --rich [View Less]

10 15

Specify number of items to allocate for array.array() constructor
by Sven Rahmann Feb. 22, 2020

Feb. 22, 2020

At the moment, the array module of the standard library allows to create arrays of different numeric types and to initialize them from an iterable (eg, another array). What's missing is the possiblity to specify the final size of the array (number of items), especially for large arrays. I'm thinking of suffix arrays (a text indexing data structure) for large texts, eg the human genome and its reverse complement (about 6 billion characters from the alphabet ACGT). The suffix array is a long int … [View More]

14 20

Implicit string literal concatenation considered harmful?
by Guido van Rossum March 14, 2018

March 14, 2018

I just spent a few minutes staring at a bug caused by a missing comma -- I got a mysterious argument count error because instead of foo('a', 'b') I had written foo('a' 'b'). This is a fairly common mistake, and IIRC at Google we even had a lint rule against this (there was also a Python dialect used for some specific purpose where this was explicitly forbidden). Now, with modern compiler technology, we can (and in fact do) evaluate compile-time string literal concatenation with the '+' … [View More]

51 165

https://docs.python.org/fr/ ?
by Julien Palard Feb. 2, 2017

Feb. 2, 2017

o/ The french translation of the Python Documentation [1][2] has translated 20% of the pageviews of docs.python.org. I think it's the right moment to push it do docs.python.org. So there's some questions ! And I'd like feedback. TL;DR (with my personal choices): - URL may be "http://docs.python.org/fr/" - For localized variations of languages we should use dash and lowercase like "docs.python.org/pt-br/" - po files may be hosted on the python's github - existing script to build … [View More]doc may be patched to build translations - each translations may crosslink to others - untranslated strings may be visually marked as so I also opened: http://bugs.python.org/issue26546. # Chronology, dependencies The only blocking decision here is the URL, (also reviewing my patch ...), with those two, translated docs can be pushed to production, and the other steps can be discussed and applied one by one. # The URL ## CCTLD vs path vs subdomain I think we should use a variation of "docs.python.org/fr/" for simplicity and clarity. I think we should avoid using CCTLDs as they're sometime hard or near impossible to obtain (may cost a lot of time), also some are expensive, so it's time and money we clearly don't need to loose. Last possibility I see is to use a subdomain, like fr.docs.python.org or docs.fr.python.org but I don't think it's the role / responsibility of the sub-domain to do it. So I'm for docs.python.org/LANGUAGE_TAG/ (without moving current documentation inside a /en/). ## Language tag in path ### Dropping the default locale of a language I personally think we should not show the region in case it's redundant: so to use "fr" instead of "fr-FR", "de" instead of "de-DE", but keeping the possibility to use a locale code when it's not redundant like for "pt-br" or "de-AT" (German ('de') as used in Austria ('AT')). I think so because I don't think we'll have a lot of locale variations (like de-AT, fr-CH, fr-CA, ...) so it will be most of the time redundant (visually heavy, longer to type, longer to read) but we'll still need some locale (pt-BR typically). ### gettext VS IETF language tag format gettext goes by using an underscore between language and locale [3] and IETF goes by using a dash [4][5]. As sphinx is using gettext, and gettext uses underscore we may choose underscore too. But URLs are not here to leak the underlying implementation, and the IETF looks like to be the standard way to represent language tags. Also I visually prefer the dash over the underscore, so I'm for the dash here. ### Lower case vs upper case local tag RFC 5646 section-2.1 tells us language tags are not case sensitive, yet ISO3166-1 recommends that country codes (part of the language tag) be capitalized. I personally prefer the all-lowercase one as paths in URLs typically are lowercase. I searched for `inurl:"pt-br"` to see if I'm not too far away from the usage here and usage seems to agree with me, although there's some "pt-BR" in urls. # Where to host the translated files Currently we're hosting the *po* files in the afpy's (Francophone association for python) [6] github [1] but it may make sense to use (in the generation scripts) a more controlled / restricted clone in the python github, at least to have a better view of who can push on the documentation. We may want to choose between aggregating all translations under the same git repository but I don't feel it's useful. # How to Currently, a python script [7] is used to generate `docs.python.org`, I proposed a patch in [8] to make this script clone and build the french translation too, it's a simple and effective way, I don't think we need more ? Any idea welcome. In our side, we have a Makefile [12] to build the translated doc which is only a thin layer on top of the Sphinx Makefile. So my proposed patch to build scripts "just" delegate the build to our Makefile which itself delegate the hard work to the Sphinx Makefile. # Next ? ## Document how to translate Python I think I can (should) write a documentation on "how to start a Python doc translation project" and "how to migrate existing [9][10][11] python doc translation projects to docs.python.org" if french does goes docs.python.org because it may hopefully motivate people to do the same, and I think our structure is a nice way to do it (A Makefile to generate the doc, all versions translated, people mainly working on latest version, scripts to propagating translations to older version, etc...). ## Crosslinking between existing translations Once the translations are on `docs.python.org`, crosslinks may be established so people on a version can be aware of other version, and easily switch to them. I'm not a UI/UX man but I think we may have a select box right before the existing select box about version, on the top-left corner. Right before because it'll reflect the path: /fr/3.5/ -> [select box fr][select box 3.5]. ## Marking as "untranslated, you can help" the untranslated paragraphs The translations will always need work to follow upstream modifications: marking untranslated paragraphs as so may transform the "Oh they suck, this paragraph is not even translated :-(" to "Hey, nice I can help translating that !". There's an opened sphinx-doc ticket to do so [13] but I have not worked on it yet. As previously said I'm real bad at designing user interfaces, so I don't even visualize how I'd like it to be. [1] http://www.afpy.org/doc/python/3.5/ [2] https://github.com/afpy/python_doc_fr [3] https://www.gnu.org/software/gettext/manual/html_node/Locale-Names.html [4] http://tools.ietf.org/html/rfc5646 [5] https://en.wikipedia.org/wiki/IETF_language_tag [6] http://www.afpy.org/ [7] https://github.com/python/docsbuild-scripts/ [8] http://bugs.python.org/issue26546 [9] http://docs.python.jp/3/ [10] https://github.com/python-doc-ja/python-doc-ja [11] http://docs.python.org.ar/tutorial/3/index.html [12] https://github.com/AFPy/python_doc_fr/blob/master/Makefile [13] https://github.com/sphinx-doc/sphinx/issues/1246 -- Julien Palard [View Less]

7 8

proposal: "python -m foo" should bind sys.modules['foo']
by Cameron Simpson Jan. 18, 2017

Jan. 18, 2017

Hello all, This is a writeup of a proposal I floated here: https://mail.python.org/pipermail/python-list/2015-August/694905.html last Sunday. If the response is positive I wish to write a PEP. Briefly, it is a natural expectation in users that the command: python -m module_name ... used to invoke modules in "main program" mode on the command line imported the module as "module_name". It does not, it imports it as "__main__". An import within the program of "module_name" makes a new … [View More]instance of the module, which causes cognitive dissonance and has the side effect that now the program has two instances of the module. What I propose is that the above command line _should_ bind sys.modules['module_name'] as well as binding '__main__' as it does currently. I'm proposing that the python -m option have this effect (python pseudocode): % python -m module.name ... runs: # pseudocode, with values hardwired for clarity import sys M = new_empty_module(name='__main__', qualname='module.name') sys.modules['__main__'] = M sys.modules['module.name'] = M # load the module code from wherever (not necessarily a file - CPython # already must do this phase) M.execfile('/path/to/module/name.py') Specificly, this would have the following two changes to current practice: 1) the module is imported _once_, and bound to both its canonical name and also to __main__. 2) imported modules acquire a new attribute __qualname__ (analogous to the recent __qualname__ on functions). This is always the conanoical name of the module as resolved by the importer. For most modules __name__ will be the same as __qualname__, but for the "main" module __name__ will be '__main__'. This change has the following advantages: The current standard boilerplate: if __name__ == '__main__': ... invoke "main program" here ... continues to work unchanged. Importantly, if the program then issues "import module_name", it is already there and the existing instance is found and used. The thread referenced above outlines my most recent encounter with this and the trouble it caused me. Followup messages include some support for this proposed change, and some criticism. The critiquing article included some workarounds for this multiple module situation, but they were (1) somewhat dependent on modules coming from a file pathname and (2) cumbersome and require every end user to adopt these changes if affected by the situation. I'd like to avoid that. Cheers, Cameron Simpson <cs(a)zip.com.au> The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. - George Bernard Shaw [View Less]

5 13

Importing public symbols and simultainiously privatizing them, is too noisy
by Rick Johnson Dec. 29, 2016

Dec. 29, 2016

Has anyone else found this to be too syntactically noisy? from module import Foo as _Foo, bar as _bar That is horrifically noisy IMO. The problem is, how do we remove the noise without sacrificing intuitiveness? My first idea was to do this: from module import_private Foo, bar And while it's self explanatory, it's also too long. So i thought... from module _import Foo, bar I'm leaning more towards the latter, but i'm not loving it either. Any ideas?

9 17

singledispatch for instance methods
by Tim Mitchell Dec. 26, 2016

Dec. 26, 2016

Hi All, We have a modified version of singledispatch at work which works for methods as well as functions. We have open-sourced it as methoddispatch (pypi: https://pypi.python.org/pypi/methoddispatch). IMHO I thought it would make a nice addition to python stdlib. What does everyone else think?

2 2

Null coalescing operator
by Arek Bulski Nov. 5, 2016

Nov. 5, 2016

Sometimes I find myself in need of this nice operator that I used back in the days when I was programming in .NET, essentially an expression >>> expr ?? instead should return expr when it `is not None` and `instead` otherwise. A piece of code that I just wrote, you can see a use case: def _sizeof(self, context): if self.totalsizeof is not None: return self.totalsizeof else: raise SizeofError("cannot calculate size") With the oprator it … [View More]

36 123

SI scale factors alone, without units or dimensional analysis
by Steven D'Aprano Oct. 29, 2016

Oct. 29, 2016

Ken has made what I consider a very reasonable suggestion, to introduce SI prefixes to Python syntax for numbers. For example, typing 1K will be equivalent to 1000. However, there are some complexities that have been glossed over. (1) Are the results floats, ints, or something else? I would expect that 1K would be int 1000, not float 1000. But what about fractional prefixes, like 1m? Should that be a float or a decimal? If I write 7981m I would expect 7.981, not 7.9809999999999999, so … [View More]maybe I want a decimal float, not a binary float? Actually, what I would really want is for the scale factor to be tracked separately. If I write 7981m * 1M, I should end up with 7981000 as an int, not a float. Am I being unreasonable? Obviously if I write 1.1K then I'm expecting a float. So I'm not *entirely* unreasonable :-) (2) Decimal or binary scale factors? The SI units are all decimal, and I think if we support these, we should insist that K == 1000, not 1024. For binary scale factors, there is the IEC standard: http://physics.nist.gov/cuu/Units/binary.html which defines Ki = 2**10, Mi = 2**20, etc. (Fortunately this doesn't have to deal with fractional prefixes.) So it would be easy enough to support them as well. (3) µ or u, k or K? I'm going to go to the barricades to fight for the real SI prefixes µ and k to be supported. If people want to support the common fakes u and K as well, that's fine, I have no objection, but I think that its important to support the actual prefixes too. (Python 3 assumes UTF-8 as the default encoding, so it shouldn't cause any technical difficulties to support µ as syntax. The political difficulties though...) (4) What about E? E is tricky if we want 1E to be read as the integer 10**18, because it matches the floating point syntax 1E (which is currently a syntax error). So there's a nasty bit of ambiguity where it may be unclear whether or not 1E is intended as an int or an incomplete float, and then there's 1E1E which might be read as 1E1*10**18 or as just an error. Replacing E with (say) X is risky. The two largest current SI prefixes are Z and Y, it seems very likely that the next one added (if that ever happens) will be X. Actually, using any other letter risks clashing with a future expansion of the SI prefixes. (5) What about other numeric types? Just because there's no syntactic support for Fraction and Decimal shouldn't mean we can't use these scale factors with them. (6) What happens to int(), float() etc? I wouldn't want int("23K") to suddenly change from being an error to returning 23000. Presumably we would want int to take an optional argument to allow the interpretation of scale factors. This gives us an advantage: int("23E", scale=True) is unambiguously an int, and we can ignore the fact that it looks like a float. (7) What about repr() and str()? I don't think that the repr() or str() of numeric types should change. But perhaps format() could grow some new codes to display numbers using either the most obvious scale factor, or some specific scale factor. * * * This leads to my first proposal: require an explicit numeric prefix on numbers before scale factors are allowed, similar to how we treat non-decimal bases. 8M # remains a syntax error 0s8M # unambiguously an int with a scale factor of M = 10**6 0s1E1E # a float 1E1 with a scale factor of E = 10**18 0s1.E # a float 1. with a scale factor of E, not an exponent int('8M') # remains a ValueError int('0s8M', base=0) # returns 8*10**6 Or if that's too heavy (two whole characters, plus the suffix!) perhaps we could have a rule that the suffix must follow the final underscore of the number: 8_M # int 8*10*6 123_456_789_M # int 123456789*10**6 123_M_456 # still an error 8._M # float 8.0*10**6 int() and float() take a keyword only argument to allow a scale factor when converting from strings: int("8_M") # remains an error int("8_M", scale=True) # allowed This solves the problem with E and floats. Its only a scale factor if it immediately follows the final underscore in the float, otherwise it is the regular exponent sign. Proposal number two: don't make any changes to the syntax, but treat these as *literally* numeric scale factors. Add a simple module to the std lib defining the various factors: k = kilo = 10**3 M = mega = 10**6 G = giga = 10**9 etc. and then allow the user to literally treat them as scale factors by multiplying: from scaling import * int_value = 8*M float_value = 8.0*M fraction_value = Fraction(1, 8)*M decimal_value = Decimal("1.2345")*M and so forth. The biggest advantage of this is that there is no syntactic changes needed, it is completely backwards compatible, it works with any numeric type and even non-numbers: py> x = [None]*M py> len(x) 1000000 You can even scale by multiple factors: x = 8*M*K Disadvantages: none I can think of. (Some cleverness may be needed to have fractional scale values work with both floats and Decimals, but that shouldn't be hard.) -- Steve [View Less]

14 20

if-statement in for-loop
by Dominik Gresch Oct. 5, 2016

Oct. 5, 2016

Hi, I've recently found myself writing code similar to this: for i in range(10): if i == 5: continue "body" which I find a bit ugly. Obviously the same could be written as for i in range(10): if i != 5: "body" but here you would have to look at the end of the body to see if something happens when i==5. So I asked myself if a syntax as follows would be possible: for i in range(10) if i != 5: body Personally, I find this extremely intuitive since this … [View More]

17 33

Suggestion: Clear screen command for the REPL
by João Matos Oct. 4, 2016

Oct. 4, 2016

Hello, I would like to suggest adding a clear command (not function) to Python. It's simple purpose would be to clear the REPL screen, leaving the >>> prompt at the top left of the screen. This is something very basic but also very useful for newbies learning Python from the REPL. After some trial and errors it is best to start with a clean screen. Clearing the screen helps clear your mind. Historically it is a common command in interpreted languages. Best regards, JM

16 52

Delay evaluation of annotations
by אלעזר Sept. 27, 2016

Sept. 27, 2016

Hi all, Annotations of function parameters and variables are evaluated when encountered. This makes it necessary to use string representation for names that are not yet bound, which affects almost every class definition. It is also easy to forget, and the result might be a (very uninteresting) exception in certain untested paths, e.g. inside functions. Editors and IDEs also don't handle it well; for example, PyDev does not consider string-annotations as an occurrence of the name, and warns … [View More]

15 72

Re: [Python-ideas] Delay evaluation of annotations
by David Mertz Sept. 25, 2016

Sept. 25, 2016

On Sep 25, 2016 11:54 AM, "אלעזר" <elazarg(a)gmail.com> wrote: > Not that I think it's a killer argument, but why a red herring? Quick search does not find such an explicit option in Gedit, PyDev and yes, Notepad++.exe. It is not a common or default option. This would equally be an argument against any syntax change, any new keyword, and likewise any new built-in. Depending on what your editor does, it would perhaps be an argument against new common names in the standard library. All … [View More]

2 1

Fwd: Re: Delay evaluation of annotations
by David Mertz Sept. 25, 2016

Sept. 25, 2016

> 1. Please consider disallowing the use of side effects of any kind in annotations, in that it is not promised when it will happen, if at all. So that a change 3 years from now will be somewhat less likely to break things. Please consider doing this for version 3.6; it is feature-frozen, but this is not (yet) a feature, and I got the feeling it is hardly controversial. What you are proposing is a very large and controversial feature change. Just repeating that you think there documented 10-… [View More]

1 0

Re: [Python-ideas] Python-ideas Digest, Vol 118, Issue 124
by Святослав Молодских Sept. 24, 2016

Sept. 24, 2016

23 сент. 2016 г. 20:00 пользователь <python-ideas-request(a)python.org> написал: > > Send Python-ideas mailing list submissions to > python-ideas(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/python-ideas > or, via email, send a message with subject or body 'help' to > python-ideas-request(a)python.org > > You can reach the person managing the list at > … [View More]python-ideas-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-ideas digest..." > > Today's Topics: > > 1. Re: Delay evaluation of annotations (Chris Angelico) > > > ---------- Пересылаемое сообщение ---------- > From: Chris Angelico <rosuav(a)gmail.com> > To: python-ideas <python-ideas(a)python.org> > Cc: > Date: Sat, 24 Sep 2016 01:58:59 +1000 > Subject: Re: [Python-ideas] Delay evaluation of annotations > On Fri, Sep 23, 2016 at 11:58 PM, אלעזר <elazarg(a)gmail.com> wrote: > > What other context you see where the result of an expression is not intended > > to be used at all? Well there's Expression statements, which are evaluated > > for side effect. There's docstrings, which are a kind of annotations. What > > else? The only other that comes to mind is reveal_type(exp)... surely I > > don't need evaluation there. > > Function annotations ARE used. They're stored as function attributes, > just as default argument values and docstrings are. (It's not the > language's problem if you never use them.) > > >> The PEP that introduced them describes them as expressions: > > > > Syntactically, yes. Just like X in "a = lambda: X" is an expression, but you > > don't see it evaluated, do you? And this is an _actual_ expression, > > undeniably so, that is intended to be evaluated and used at runtime. > > And the X in "if False: X" is a statement, but you don't see it > evaluated either. This is an actual expression that has to be > evaluated and used just like any other does. > > >> I think its time to give up arguing that annotations aren't expressions. > >> > > > > I don't care if you call them expressions, delayed-expressions, or flying > > monkeys. The allowed syntax is exactly that of an expression (like inside a > > lambda). The time of binding of names to scope is the same (again like a > > lambda) but the evaluation time is unknown to the non-reflecting-developer. > > Decorators may promise time of evaluation, if they want to. > > Thing is, literally every other expression in Python is evaluated at > the point where it's hit. You can guard an expression with control > flow statements or operators, but other than that, it will be hit when > execution reaches its line: > > def func(x): > expr # evaluated when function called > > if cond: > expr # evaluated if cond is true > > [expr for x in range(n)] # evaluated if n > 0 > (expr for x in [1]) # evaluated when genexp nexted > expr if cond else "spam" # evaluated if cond is true > lambda: expr # evaluated when function called > > def func(x=expr): pass # evaluated when function defined > def func(x: expr): pass # evaluated when function defined > > Default arguments trip some people up because they expect them to be > evaluated when the function's called, but it can easily be explained. > Function annotations are exactly the same. Making them magically > late-evaluate would have consequences for the grokkability of the > language - they would be special. Now, that can be done, but as > Rumplestiltskin keeps reminding us, all magic comes with a price, so > it has to be strongly justified. (For instance, the no-arg form of > super() is most definitely magical, but its justification is obvious > when you compare Py2 inheritance with Py3.) > > > "Unknown evaluation time" is scary. _for expressions_, which might have side > > effects (one of which is running time). But annotations must be pure by > > convention (and tools are welcome to warn about it). I admit that I propose > > breaking the following code: > > > > def foo(x: print("defining foo!")): pass > > > > Do you know anyone who would dream about writing such code? > > Yes, side effects make evaluation time scary. But so do rebindings, > and any other influences on expression evaluation. Good, readable code > generally follows the rule that the first instance of a name is its > definition. That's why we put imports up the top of the script, and so > on. Making annotations not work that way isn't going to improve > readability; you'd have to search the entire project for the class > being referenced. And since you can't probe them at definition time, > you have to wait until, uhh, SOME time, to do that search - you never > know where the actual name binding will come from. (It might even get > injected from another file, so you can't statically search the one > file.) > > >> You want to make them fancy, give them super-powers, in order to solve > >> the forward reference problem. I don't think that the problem is serious > >> enough to justify changing the semantics of annotation evaluation and > >> make them non-standard, fancy, lazy-evaluated expressions. > >> > > > > My proposal solves the forward reference problem, but I believe in it > > because I believe it is aligned with what the programmer see. > > This is on par with a proposal to make default argument values > late-bind, which comes up every now and then. It's just not worth > making these expressions magical. > > >> > > class MyClass: > >> > > pass > >> > > > >> > > def function(arg: MyCalss): > >> > > ... > >> > > > >> > > I want to see an immediate NameError here, thank you very much > >> > > >> > Two things to note here: > >> > A. IDEs will point at this NameError > >> > >> Some or them might. Not everyone uses an IDE, it is not a requirement > >> for Python programmers. Runtime exceptions are still, and always will > >> be, the primary way of detecting such errors. > > > > How useful is the detection of this error at production? > > The sooner you catch an error, the better. Always. > > > Can you repeat that? NameError indeed happens at runtime, but the scope in > > which MyCalss was looked up for is determined at compile time - as far as I > > know. The bytecode-based typechecker I wrote rely on this information being > > accessible statically in the bytecode. > > > > def foo(): > > locals()['MyType'] = str > > def bar(a : MyType): pass > > > >>>> foo() > > Traceback (most recent call last): > > File "<stdin>", line 1, in <module> > > File "<stdin>", line 4, in foo > > NameError: name 'MyType' is not defined > > > > What do I miss? > > That locals() is not editable (or rather, that mutations to it don't > necessarily change the actual locals). This is equivalent to: > > def foo(): > locals()['MyType'] = str > print(MyType) > > >>> foo() > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "<stdin>", line 3, in foo > NameError: name 'MyType' is not defined > > In each case, you have to *call* foo() to see the NameError. It's > happening at run time. > > > This way or the other, the very least that I hope, is explicitly forbidding > > reliance on side-effect or any other way to distinguish evaluation time of > > annotation expressions. Annotations must be pure, and the current promise of > > evaluation time should be deprecated. > > Define "pure". Function decorator syntax goes to some lengths to > ensure that this is legal: > > @deco(arg) > def f(): pass > > PEP 484 annotations include subscripting, even nested: > > def inproduct(v: Iterable[Tuple[T, T]]) -> T: > > so you'd have to accept some measure of run-time evaluation. > > It's worth reiterating, too, that function annotations have had the > exact same semantics since Python 3.0, in 2008. Changing that now > would potentially break up to eight years' worth of code, not all of > which follows PEP 484. When Steve mentioned 'not breaking other uses > of annotations', he's including this large body of code that might > well not even be visible to us, much less under python.org control. > Changing how annotations get evaluated is a *major, breaking change*, > so all you can really do is make a style guide recommendation that > "annotations should be able to be understood with minimal external > information" or something. > > > Additionally, before making it impossible to go back, we should make the new > > variable annotation syntax add its annotations to a special object > > __reflect__, so that __reflect__.annotations__ will allow forcing evaluation > > (since there is no mechanism to do this in a variable). > > Wow, lots of magic needed to make this work. Here's my > counter-proposal. In C++, you can pre-declare a class like this: > > class Mutual2; //Pre-declare Mutual2 > class Mutual1 { > Mutual2 *ptr; > }; > class Mutual2 { > Mutual1 *ptr; > } > > Here's how you could do it in Python: > > Mutual2 = "Mutual2" # Pre-declare Mutual2 > class Mutual1: > def spam() -> Mutual2: pass > class Mutual2: > def spam() -> Mutual1: pass > > Problem solved, no magic needed. > > ChrisA > > > _______________________________________________ > Python-ideas mailing list > Python-ideas(a)python.org > https://mail.python.org/mailman/listinfo/python-ideas > [View Less]

1 0

from __pip__ import
by אלעזר Sept. 21, 2016

Sept. 21, 2016

Many proposals to add something to stdlib are rejected here with the suggestion to add such library to pypi first. As noted by someone, pypi is not as reachable as stdlib, and one should install that package first, which many people don't know how. Additionally, there is no natural distinction between 3rd party dependencies and in-project imports (at least in tiny projects). This can be made easier if the first line of the program will declare the required library, and executing it will try to … [View More]

15 60

Make partial a built-in
by Michel Desmoulin Sept. 21, 2016

Sept. 21, 2016

While lambda are a great tools that can do the job of partial, partial is always a superior solution when trying to curry a Python function because it's explicit and easier to debug. Having to import it everytime means I usually go for a lambda out of lazyness, but also means that most people don't know about it. If think having it in the builtins would promote cleaner, faster code, but would also facilitate and encourage coding patterns using curried functions.

15 28

Overloading operators for testing
by Arek Bulski Sept. 21, 2016

Sept. 21, 2016

I am using declarative testing a lot and I found out why unit tests are so clunky. The reason why assertEquals(a,b) is used is because if we put `assert a==b` then nose can catch the AssertionError but wont find out what was returned or expected. This could be easily overcome if we allow oveloading == operator from outside. Right now == would have to be changed for every lefhand object that is compared in the tests, builtin types including. We could use a way to change it from above, so to … [View More]

6 5

___repr__ on collections.UserDict, collections.UserList, collections.UserString
by David Kendal Sept. 19, 2016

Sept. 19, 2016

Hi, the UserDict, UserList, and UserString classes in the collections module ought to provide __repr__ implementations along the following lines: def __repr__(self): return '%s(%r)' % (type(self).__name__, self.data) so that they give useful immediate information in the repl to the effect that they are not mere dicts/lists/strs but user-defined container types. dpk (David P. Kendal) · Nassauische Str. 36, 10717 DE · http://dpk.io/ The reason we had no idea how cats worked was because,… [View More]

1 0

divmod(): fallback to __floordiv__ and __mod__?
by Spencer Brown Sept. 19, 2016

Sept. 19, 2016

Currently, calling divmod() on a class with __floordiv__ and __mod__ defined, but not __divmod__ raises a TypeError. Is there any reason why it doesn't fallback to (self // x, self % x)? The invariants described in the function documentation should still apply (with well behaved % and //). If divmod() doesn't make sense setting it to None should be permitted. - Spencer

6 9

(Windows-only - calling Steve Dower) Is Python for Windows using PGO? If not consider this a suggestion.
by João Matos Sept. 18, 2016

Sept. 18, 2016

Hello, Is Python for Windows using PGO (Profile Guided Optimizations)? If not consider this a suggestion. Best regards, JM

2 2

(Windows-only - calling Steve Dower) Consider adding a symlink to pip in the same location as the py launcher
by João Matos Sept. 18, 2016

Sept. 18, 2016

Hello, If Py3.5 is installed in user mode instead of admin (all users) and we follow your advice that we shouldn't add it to the PATH env var, we can execute Python using the py launcher, but we can't use pip. Please consider adding a pip symlink in the same location as the py launcher. Best regards, JM

3 4

typing.modifiers
by אלעזר Sept. 17, 2016

Sept. 17, 2016

With the new variable annotation syntax, it is possible to implement a useful "modifiers" library, all used as superclasses. Possible modifiers: * Named / Struct: annotation-defined fields. No monkey patching. * Immutable / Const * Sealed / Final: unsubclassable class * Array: a mutable tuple. (No "+" or "*" operators) There of course may be others. Subclasses of the modifiers get default methods and behavior. The modifiers can used in combinations. For example: class Command(… [View More]Immutable, Named): cmd : str = "NOP" value : int = None load = Command(cmd="load", value=1000) # no positional arguments The syntax for Array is less pretty: class Command(Array): _0 : str = "NOP" _1 : int = None load = Command("load", 1000) # no positional arguments Many other combinations are useful too. "Sealed" is almost orthogonal to the others, Obviously, some combinations already exists: * (Immutable, Array) is similar to Tuple * NamedTuple is another name for (Immutable, Named, Array). * Enum should be on this list too Some less promising modifiers: * Namespace / Static: uninstantiable class. A module. * Volatile, for externally facing classes. Hints the static checkers / jitters that they should not assume they know the values. * Abstract Alternatives and cons : I suggest base classes instead of decorators since NamedTuple and tuple go this way, and since it is static information. I am not sure which is better though. Metaclass parameters can be used (and actually used in my implementation, and in NamedTuple's), but the syntax is uglier. "array" is only for numeric values in Python, so this name is problematic.So is struct. I have a rough implementation for most of the above; much of it is not hard in general, though some details are hard to get right. --- Benefits of putting such a collection in stdlib (instead of as an external package) include: 1. This information can be used by typecheckers, and also by users, to reason about programs. If isinstance(x, ImmutableArray), then x is an instantiation of ImmutableArray. 2. A conventional syntax and a single answer for "How do I make my class immutable", "How do I make my class unsubclassable" 3. The syntax, especially for Struct as above, is pretty and clean. The Array syntax is less so. 4. I think that the array implementation can use internal CPython details to be implemented efficiently. I am not sure that typing.modifiers is the right place, since these are not exactly type hints; they generate methods, and are intended to be enforced at runtime. I think that even if this idea is not accepted, the general theme is something that might be useful to keep in mind, stdlib might accumulate such modifiers, and it will be nice to keep things uniform. Opinions? ~Elazar [View Less]

5 9

Adding bytes.frombuffer() constructor
by INADA Naoki Sept. 16, 2016

Sept. 16, 2016

bytearray is used as read / write buffer in asyncio. When implementing reading stream, read() API may be defined as returns "bytes". To get front n bytes from bytearray, there are two ways. 1. bytes(bytearray[:n]) 2. bytes(memoryview(bytearray)[:n]) (1) is simplest, but it produces temporary bytearray having n bytes. While (2) is more efficient than (1), it uses still temporary memoryview object, and it looks bit tricky. I want simple, readable, and temporary-less way to get part of … [View More]

3 4

Naming convention for Enums
by Bar Harel Sept. 15, 2016

Sept. 15, 2016

Hey guys, This is something that has been discussed before several times, whether in Python mailing list and whether in stackoverflow, each providing different answers. I believe that we should add a naming convention in pep8 to solve this issue. Enums in Python are a unique creature - it's a class, the members aren't injected into module level, it can have methods, but it's attributes are somewhat consts (although they can be modified via __new__). The fact that currently every person … [View More]

6 11

Generics Syntax
by אלעזר Sept. 15, 2016

Sept. 15, 2016

This suggestion is so obvious that it's likely has been discussed, but I can't find any reference (It's not what PEP-3124 talks about). Generic class syntax, now: _T_co = TypeVar('_T', covariant=True) class Container(Generic[_T_co]): @abstractmethod def __contains__(self, x: _T_co) -> bool: ... (yes it's object in reality) Generic class syntax, suggested: class Container[+T]: @abstractmethod def __contains__(self, x: T) -> bool: ... The + … [View More]

3 7

New Python syntax for continuing definitions for existing classes
by Pim Schellart Sept. 14, 2016

Sept. 14, 2016

Dear Python Developers, we have a potential idea for enhancing Python. Below you will find what the PEP might look like. A reference implementation has been written and will be posted in a follow up message. We are look forward to hearing your feedback and ideas. Kind regards, Pim Schellart & Nate Lust Abstract ======== This PEP proposes the introduction of new syntax to create a community standard, readable way to continue a definition for classes which are already defined. … [View More]Rationale ========= Extending existing classes with new members currently requires the members to be defined outside of class scope and then assigned to the class, as in:: def foo(self): pass A.foo = foo del foo This pattern frequently occurs when extending classes from extension modules written in other languages, but is also not uncommon in pure Python code. This syntax is however cumbersome and error prone for the following reasons: 1. The name of the new member might clash with an existing name at module scope. 2. The developer might forget the assignment. 3. The developer might forget to delete the temporary, leaving it at module scope where it is meaningless (or worse non functional as a stand alone function). Alternatives are to use inheritance, lambda or a decorator. Inheritance is not a good option for use with Python extension modules (written in for instance C / C++). The reason is that if in the other language the inheritance relation is A<-B, and this is exposed to Python, we can't add functionality to A in Python by subclassing A<-A' without also introducing B' that inherits from B and A'. Thus one change propagates all the way down the inheritance chain. Inheritance with pure Python classes exhibit the same issue. It might be tempting to avoid (1) and (2) by subclassing, reusing the name of the base class. However this approach suffers from two issues. The first issue, like above, has to do with the inheritance relation. If A subclasses A and B subclasses A (A<-A<-B), then calls to super in B may have unexpected behavior (anything using the method resolution order may exhibit the same unexpectedness). The second issue arises when class B inherits from class A before it is extended (A<-B) and then class A is extended though inheritance (A<-A). Class A will now have new methods and data members which are not present in B, as it is a subclass only of the original A. This would be confusing to anyone examining B as it would be apparently missing members of it's parent. Adding attributes to a class using lambda is not equivalent because it only allows for expressions and not statements. A class decorator (say "@continue_class(A)" that takes methods and attributes from the class and attaches them to the wrapped type) works, but is cumbersome and requires a new class name to be introduced for each extended type. Thus suffering from the same problems numbered (1) and (2) above. A function decorator could extend a class, but also requires defining a new name and suffers from problems (1) and (2). This proposal adds the keyword combination "continue class" which instructs the interpreter to add members to an existing class. This syntax requires no new keywords and allows all existing Python code to continue to function with no modification. By combining existing keywords it avoids name clashes. Precedents for this exists in "is not" and "yield from". Another big advantage of the "continue class" syntax is discoverability. For humans, the meaning of the syntax is evident, the class will be continued with the following block. Automated tools will also benefit from this syntax as parsing a file to discover the complete definition of a class will be easier than trying to detect 'monkey patching'. Semantics ========= The following two snippets are semantically identical:: continue class A: x = 5 def foo(self): pass def bar(self): pass def foo(self): pass def bar(self): pass A.x = 5 A.foo = foo A.bar = bar del foo del bar Alternatives ============ An alternative could be to allow a function definition to include a class specifier as in:: def A.foo(self): pass Implementation ============== Adapting Python's grammar to support the continue class keyword combination requires modifying the grammar file to parse for the new keyword combination. Additional changes to Python.asdl, ast.c, compile.c, symtable.c will be required to process the parsed syntax to byte code. A reference implementation (already written and posted separately) parses the continue class line for the name of the class to be modified and loads it. Next the body of the block is compiled into a code object scoped by the name of the class. The class and the code are used as arguments to a new built-in module, called __continue_class__. This function evaluates the code block, passing the results to setattr on the supplied class. [View Less]

10 15

An exciting opportunity to update PEP 3156
by Yury Selivanov Sept. 13, 2016

Sept. 13, 2016

Guido and I are looking for help with PEP 3156. Currently it's out of sync with the docs. It would be cool if someone could volunteer and update it (at least make sure that all APIs are up to date). Yury

6 6

Fwd: Null coalescing operator
by David Mertz Sept. 13, 2016

Sept. 13, 2016

Sorry, I sent this accidentally as private reply, then tried to fix it on phone. The latter produced horrible formatting. Please just read this version. On Sat, Sep 10, 2016 at 4:10 PM, Guido van Rossum <guido(a)python.org> wrote: > So you're offering `NoneCoalesce(x).bar` as less-ugly alternative to > `x?.bar`... Color me unconvinced. > No, I'm offering a more realistic use pattern: for x in get_stuff(): x = NoneCoalesce(x) # ... bunch of stuff with x ... # ..… [View More]

15 45

Re: [Python-ideas] Changing optimisation level from a script
by Damien George Sept. 13, 2016

Sept. 13, 2016

Thanks all for the input on this topic. Let me summarise what was said. There was discussion that the ability to change the "optimize" value is not enough and there should be more fine grained control over other flags/settings. Perhaps that is true, but at least for our case in MicroPython there's only one (runtime) optimisation variable and it mimics CPython's -O command line option (affecting __debug__, assert compilation, and whether line numbers are included in the bytecode). … [View More]

2 1

Re: [Python-ideas] Changing optimisation level from a script
by Damien George Sept. 13, 2016

Sept. 13, 2016

Hi Petr, > The API you proposed here comes is similar to something I see a lot in > MicroPython libraries: functions/methods that combine a getter and setter. > For example, to set the value on a pin, you do: > pin.value(1) > and to read, you do: > result = pin.value() > > If an API like this was added to the stdlib, I'd expect it to use a > property, e.g. > pin.value = 1 > result = pin.value > > > I was wondering, what's the story … [View More]

1 0

Re: [Python-ideas] [Python-Dev] Drastically improving list.sort() for lists of strings/ints
by Tim Peters Sept. 13, 2016

Sept. 13, 2016

[Elliot Gorokhovsky <elliot.gorokhovsky(a)gmail.com>] > Wow, Tim himself! And Elliot himself! It's a party :-) > Regarding performance on semi-ordered data: we'll have to > benchmark to see, but intuitively I imagine radix would meet Timsort > because verifying that a list of strings is sorted takes Omega(nw) > (which gives a lower bound on Timsort), where w is the word length. Don't take that stuff so seriously. The number of character comparisons needed when … [View More]comparing two strings is _typically_ small. Consecutive corresponding characters are compared only until the first non-equal pair is found. For example, if the alphabet is 256 characters, for random strings it only requires one character comparison 255 of 256 times on average (because there's only 1 chance in 256 that the first characters _are_ equal). > Radix sort is Theta(nw). So at least asymptotically it checks out. Analogously, most-significant-byte-first radix sort stops on the pass after the longest common prefix is processed. But the real devils are in the details. More on that below. > I think if one uses the two-array algorithm, other semi-sortings can also > be exploited, since the items get placed into their respective buckets > in the order in which they appear in the list. So, for the example you gave, > one pass would sort it correctly Can't know that unless you specify which algorithm you have in mind. If you're talking about the one in the paper, _any_ sequence with values all from range(256) will be sorted in the first pass, because each value would get its own bucket. > (since the list has the property if x_1 and x_2 are in bucket b, x1 comes > before x2 in the list, so x1 will also come before x2 in the bucket. Except > possibly for one "border bucket" that includes n/2). And then > it would just be Theta(nw/b) in each bucket to verify sorted. I don't think you want to go there. If you're keen on radix sorting, stick to radix sorting until it's as good as you can make it. "To verify sorted" requires the (potentially) arbitrarily expensive kinds of comparisons you're trying to _eliminate_. That's not saying there's no possible advantage from doing some of each; it's warning that pursuing every off-target idea that comes up will leave you spinning in circles. > I mean honestly the cool thing about radix is that the best case for > Timsort on strings, Omega(nw), is the worst case for radix! "Details": I wrote a prototype radix sort along the lines of the paper's 2-array version, in Python. When there's "a real" O() improvement to be had, one can usually demonstrate it by a Python program incorporating the improvement even though it's fighting against (unimproved) C code. For example, I helped someone here craft a Python radix sort for integers that runs faster than list.sort(), although it required a list over 10,000,000 elements and a radix of at least 4096 ;-) http://stackoverflow.com/questions/20207791/pushing-radix-sort-and-python-t… However, I'm not having that kind of luck with strings. Here's a detail that bites: as above, string comparisons _usually_ get out fast, after comparing just a few characters. So to construct a bad case for list.sort() in this respect, I want lots of strings with a long common prefix. For example, suppose we have a list of a million strings each of which starts with "A"*20000 (20 thousand "A"). String comparisons are then very expensive (each one requires at least 20001 character comparisons to resolve). But that also makes the radix sort excruciatingly slow! On the first pass, it copies the million elements, puts them all in the same bucket, then copies them all back to exactly where they started from. Pass 2, exactly the same, but looking at the 2nd occurrence of "A" in each string. Ditto for pass 3, pass 4, and ... pass 20,000. By then we've copied a million elements 40,000 times - but still have exactly the same array we started with. The data-copying costs are killing it (yes, it's only copying pointers, but 40 billion pointer copies aren't cheap - and on my 64-bit box, amounts to copying 320 billion bytes). On the same data, list.sort() is doing a great many character comparisons, but doing _far_ less pointer copying, and that leaves it running circles around the radix sort - list.sort() can't copy pointers more than about N*log2(N) times, and log2(1000000) is much smaller than 40,000. To be complete, I'll attach the code I used. Note that the paper is only concerned with "C strings": 0 bytes are special, acting as string terminators. 0 bytes aren't special in Python, so we need 257 buckets. I'm also using Python 3, where strings are always Unicode, and code points (`ord()` results) can be as large as 1,114,111. So this radix sort can't work directly on Python 3 strings; instead it expects a list of `bytes` (or `bytearray`) objects. def rsort(xs): count = [0] * 257 bindex = [None] * 257 stack = [(0, len(xs), 0)] # start index, length, byte index push = stack.append pop = stack.pop while stack: assert all(c == 0 for c in count) si, n, bi = pop() txs = xs[si: si + n] #if n < 30: # xs[si: si + n] = sorted(txs) # continue minbyte, maxbyte = 255, 0 for x in txs: b = x[bi] if bi < len(x) else -1 count[b] += 1 if count[b] == 1 and b >= 0: if b < minbyte: minbyte = b if b > maxbyte: maxbyte = b bindex[-1] = si sofar = si + count[-1] count[-1] = 0 for b in range(minbyte, maxbyte + 1): c = count[b] if c == 0: continue if c > 1: push((sofar, c, bi + 1)) bindex[b] = sofar sofar += c count[b] = 0 assert sofar == si + n for x in txs: b = x[bi] if bi < len(x) else -1 xs[bindex[b]] = x bindex[b] += 1 A little utility to generate "random" input may also be helpful: def build(n, m): from random import randrange xs = [] for _ in range(n): xs.append(bytes(randrange(256) for _ in range(randrange(m)))) return xs Then, e.g., >>> xs = build(1000000, 20) >>> ys = xs[:] >>> rsort(xs) >>> assert xs == sorted(ys) [View Less]

1 0

Re: [Python-ideas] [Python-Dev] Drastically improving list.sort() for lists of strings/ints
by Tim Peters Sept. 11, 2016

Sept. 11, 2016

[redirected from python-dev, to python-ideas; please send followups only to python-ideas] [Elliot Gorokhovsky <elgo8537(a)colorado.edu>] > ... > TL;DR: Should I spend time making list.sort() detect if it is sorting > lexicographically and, if so, use a much faster algorithm? It will be fun to find out ;-) As Mark, and especially Terry, pointed out, a major feature of the current sort is that it can exploit many kinds of pre-existing order. As the paper you referenced says, "… [View More]

1 0

Re: [Python-ideas] Shuffled
by Arek Bulski Sept. 11, 2016

Sept. 11, 2016

> And if random.shuffle() returned a new list, other people would be bitten because they expected it to be in-place. No one proposes that. shuffled() should be a new function. > One moderately stong piece of evidence would be if this function is widely available in third-party libraries and other languages. Wrong. Python is NOT known for doing everything by how it was done before. My father used statically typed variables and his father used statically typed variables so I will use... … [View More]

15 38

Fwd: Re: Null coalescing operator
by David Mertz Sept. 10, 2016

Sept. 10, 2016

On Sat, Sep 10, 2016 at 4:10 PM, Guido van Rossum <guido(a)python.org> wrote: > > So you're offering `NoneCoalesce(x).bar` as less-ugly alternative to > `x?.bar`... Color me unconvinced. No, I'm offering a more realistic use pattern: > for x in get_stuff(): > > x = NoneCoalesce(x) > > # ... bunch of stuff with x ... > # ... more stuff with nested keys or attributes ... > > x2 = x.foo > > x3 = x.bar.baz[x2] > > … [View More] x4 = x(x.val) > > result = x3(x4) > > As a less ugly alternative in the fairly uncommon case that you want None coalescing as the behavior of getting attributes, keys, call values, etc. that may or may not be available (AND where you don't want to wrap all of those access patterns in one try/except block). In contrast, the ugly version of even this pretty simple toy code with the hypothetical syntax would be: > for x in get_stuff(): > > # ... bunch of stuff with x ... > > # ... more stuff with nested keys or attributes ... > > x2 = x?.foo > > x3 = x?.bar?.baz?[x2] > > x4 = x?(x?.val) > > result = x3?(x4) This second case looks absolutely awful to me. And real world uses, if implemented, would quickly get much worse than that. > > On Sat, Sep 10, 2016 at 4:06 PM, David Mertz <mertz(a)gnosis.cx> wrote: > > Actually, I guess the example I liked was from the year ago discussion. And > > it didn't do *exactly* what I think a wrapper should. What I'd want would > > be like this: > > > > class NoneCoalesce(object): > > "Standard operations on object for 'is not None'" > > def __init__(self, obj): > > self.obj = obj > > > > def __getattr__(self, name): > > try: > > return getattr(self.obj, name) > > except AttributeError: > > return NoneCoalesce(None) > > > > def __getitem__(self, item): > > try: > > return self.obj[item] > > except (TypeError, KeyError): > > return NoneCoalesce(None) > > > > def __call__(self, *args, **kwds): > > try: > > return self.obj(*args, **kwds) > > except TypeError: > > return NoneCoalesce(None) > > > > def __bool__(self): > > return self.obj is not None > > > > def __repr__(self): > > return "NoneCoalesce[%r]" % self.obj > > > > def __str__(self): > > return "NoneCoalesce[%r]" % self.obj > > > > def __len__(self): > > try: > > return len(self.obj) > > except TypeError: > > return 0 > > > > > > Then we might use it similar to this: > > > >>>> from boltons.dictutils import OrderedMultiDict > >>>> from NoneCoalesce import NoneCoalesce > >>>> omd = OrderedMultiDict() > >>>> omd['a'] = 1 > >>>> omd['b'] = 2 > >>>> omd.add('a', 3) > >>>> nc = NoneCoalesce(omd) > >>>> nc or "Spanish Inquisition" > > Out[8]: NoneCoalesce[OrderedMultiDict([('a', 1), ('b', 2), ('a', 3)])] > >>>> nc.spam or "Spam" > > Out[9]: 'Spam' > >>>> nc['nope'].bar.baz() > > Out[10]: NoneCoalesce[None] > >>>> nc['a'] > > Out[11]: 3 > >>>> nc.getlist('a') > > Out[12]: [1, 3] > > > > > > Nothing special about boltons' OrderedMultiDict here, just something I've > > been playing with that has some distinctive methods. > > > > The idea is that we can easily have both "regular" behavior and None > > coalescing just by wrapping any objects in a utility class... and WITHOUT > > adding ugly syntax. I might have missed some corners where we would want > > behavior wrapped, but those shouldn't be that hard to add in principle. > > > > On Sat, Sep 10, 2016 at 3:21 PM, David Mertz <mertz(a)gnosis.cx> wrote: > >> > >> I find the '?.' syntax very ugly, much more so in the examples of chained > >> attributes. > >> > >> A much better way to handle the use case is to wrap objects in a class > >> that gives this "propagating None" behavior with plain attribute access. A > >> nice implementation was presented in this thread. > >> > >> > >> On Sep 10, 2016 3:16 PM, "Random832" <random832(a)fastmail.com> wrote: > >>> > >>> On Sat, Sep 10, 2016, at 13:26, Guido van Rossum wrote: > >>> > The way I recall it, we arrived at the perfect syntax (using ?) and > >>> > semantics. The issue was purely strong hesitation about whether > >>> > sprinkling ? all over your code is too ugly for Python > >>> > >>> I think that if there's "strong hesitation" about something being "too > >>> ugly" it can't really be described as "the perfect syntax". IIRC there > >>> were a couple alternatives being discussed that would have reduced the > >>> number of question marks to one [or one per object which might be None]. > >>> _______________________________________________ > >>> Python-ideas mailing list > >>> Python-ideas(a)python.org > >>> https://mail.python.org/mailman/listinfo/python-ideas > >>> Code of Conduct: http://python.org/psf/codeofconduct/ > > > > > > > > > > -- > > Keeping medicines from the bloodstreams of the sick; food > > from the bellies of the hungry; books from the hands of the > > uneducated; technology from the underdeveloped; and putting > > advocates of freedom in prisons. Intellectual property is > > to the 21st century what the slave trade was to the 16th. > > > > _______________________________________________ > > Python-ideas mailing list > > Python-ideas(a)python.org > > https://mail.python.org/mailman/listinfo/python-ideas > > Code of Conduct: http://python.org/psf/codeofconduct/ > > > > -- > --Guido van Rossum (python.org/~guido) -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. [View Less]

1 0

Changing optimisation level from a script
by Damien George Sept. 10, 2016

Sept. 10, 2016

Hi all, When starting CPython from the command line you can pass the -O option to enable optimisations (eg `assert 0` won't raise an exception when -O is passed). But, AFAIK, there is no way to change the optimisation level after the interpreter has started up, ie there is no Python function call or variable that can change the optimisation. In MicroPython we want to be able to change the optimisation level from within a script because (on bare metal at least) there is no analog of passing … [View More]

11 13

Expose reasons for SSL/TLS cert verification failures
by Chi Hsuan Yen Sept. 9, 2016

Sept. 9, 2016

Hi Python enthusiasts, Currently _ssl.c always reports CERTIFICATE_VERIFY_FAILED for any certification verification errors. In OpenSSL, it's possible to tell from different reasons that lead to CERTIFICATE_VERIFY_FAILED. For example, https://expired.badssl.com/ reports X509_V_ERR_CERT_HAS_EXPIRED, and https://self-signed.badssl.com/ reports X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT. Seems CPython does not expose such information yet? I hope it can be added to CPython. For example, creating a new … [View More]

2 2

Re: [Python-ideas] Typecheckers: there can be only one
by Hugh Fisher Sept. 9, 2016

Sept. 9, 2016

> From: Steven D'Aprano <steve(a)pearwood.info> > > I fear that you haven't grasped the fundamental difference between > gradual static typing of a dynamically-typed language like Python, and > non-gradual typing of statically-typed languages like C, Haskell, Java, > etc. Your statement seems like a reasonable fear if you think of static > typing as a mandatory pre-compilation step which prevents the code from > compiling or running if it doesn't pass. But it makes … [View More]

1 0

shuffled as a way to shuffle an iterable
by Xavier Combelle Sept. 8, 2016

Sept. 8, 2016

When thinking about the shuffled thread, it occurred to me that it was quite easy to pass an iterable and expect the iterable to be shuffled. but two mentioned implementation are close to success but fail by not taking this use case in account: def shuffled1(iterable): result = iterable[:] random.shuffle(result) return result This one might fail because an iterable don't have subscript, but in the case of range() it has. In this case it fail at the affectation place def … [View More]

2 1

kw to be ordered dict
by Arek Bulski Sept. 8, 2016

Sept. 8, 2016

I have a piece of code that essentially reimplements OrderedDict. I have been wondering why the key ordered is not preserved correctly (tests fail). c = Container(a=1,b=2,c=3,d=4) self.assertEqual(c.keys(), ["a","b","c","d"]) self.assertEqual(c.values(), [1,2,3,4]) self.assertEqual(c.items(), [("a",1),("b",2),("c",3),("d",4)]) Then I looked at collections.OrderedDict ctor docstring and it says "Initialize an ordered dictionary. The signature is the same as regular dictionaries, but keyword … [View More]

4 3

Three ways of paths canonization
by Serhiy Storchaka Sept. 8, 2016

Sept. 8, 2016

Before removing provisional state from the pathlib module, we should resolve the issue with Path.resolve(). It corresponds to os.path.realpath(), but behaves differently in case of non-existent path. Actually we can't say that any of these functions is wrong. Both behaviors make sense in different situations. The readlink utility from GNU coreutils has three mode for resolving file path: -f, --canonicalize canonicalize by following every symlink in every component … [View More]

5 4

Re: [Python-ideas] Shuffled
by Arek Bulski Sept. 8, 2016

Sept. 8, 2016

Hey, I just did put my thoughts on the discussion floor. Dont yell at me for it not being the most interesting idea ever. There were few interesting responses after all. Lets move on. pozdrawiam, Arkadiusz Bulski

1 0

Typecheckers: there can be only one
by Hugh Fisher Sept. 8, 2016

Sept. 8, 2016

There's been discussion here and on python-dev regarding PEP 526 that assumes there will be multiple type checkers for Python. I really can't see this happening. If typing annotations become common or maybe considered best practice, the Zen of Python "there should be one-- and preferably only one --obvious way to do it" will take effect. Firstly, the interpreter will need to have type checking built in. Just about every intro book and tutorial for Python says how great it is that you don't … [View More]

7 13

Re: [Python-ideas] Shuffled
by Arek Bulski Sept. 8, 2016

Sept. 8, 2016

> So, why can't you call random.shuffle(all_tests) if you want to run your tests in random order? I dont randomize test order. People should stop assuming that they know better. I need to randomize some arguments for one particular test and I cannot call shuffle between tests. Its a continous list of declarative tests. Needs to be shuffled(). https://github.com/construct/construct/blob/master/tests/test_all.py See? No way to put imperative code between tests. > sample(container, len(… [View More]

7 6

Re: [Python-ideas] Shuffled
by Arek Bulski Sept. 8, 2016

Sept. 8, 2016

> You've probably spent way more time debating shuffled here than you would have needed to add a shuffled function to your test file. Another presupposition. I did add it to my utility. I am proposing to add this to the std library so that others dont have to. pozdrawiam, Arkadiusz Bulski

2 1

Re: [Python-ideas] Shuffled
by Arek Bulski Sept. 8, 2016

Sept. 8, 2016

If you want to see the declarative tests, here it is. https://github.com/construct/construct/blob/master/tests/test_all.py pozdrawiam, Arkadiusz Bulski 2016-09-08 2:13 GMT+02:00 Arek Bulski <arek.bulski(a)gmail.com>: > See for yourself. There is a long list of declarative tests. > https://github.com/construct/construct/blob/master/tests/test_all.py > > pozdrawiam, > Arkadiusz Bulski > > > 2016-09-08 1:28 GMT+02:00 Alexander Belopolsky < > alexander.… [View More]

2 2

Re: [Python-ideas] Shuffled
by Arek Bulski Sept. 7, 2016

Sept. 7, 2016

Thank you all for the meta discussion. I know we sidetracked a bit but please everyone remember, this is an open discussion floor. I can present you a use case, just didnt think I even needed one. This seems so obvious to me that it should just go in without much saying. In the project I maintain (construct) there are declarative testcases that look like a long list of (func, args, excepted output, error type) tuples. There is no way for me to call shuffle in there. I can only use shuffled. So … [View More]

2 1

Re: [Python-ideas] PEP 530: Asynchronous Comprehensions
by Adam Bartoš Sept. 7, 2016

Sept. 7, 2016

Hello, will await be allowed also in the "if" part of comprehensions? And what about the "in" part? (e.g. if I'm not mistaken, we may have an asynchronous function returning an asynchronous iterator.) Regards, Adam Bartoš

4 5

PEP 530: Asynchronous Comprehensions
by Yury Selivanov Sept. 7, 2016

Sept. 7, 2016

Hi, Below is a proposal to add support for asynchronous comprehensions and asynchronous generator expressions in Python 3.6. I have a half-working implementation of the proposal which fully implements all required grammar and AST changes. What's left is to update the compiler to emit correct opcodes for async comprehensions. I'm confident that we can have a fully working patch before the feature freeze. Thank you, Yury PEP: 530 Title: Asynchronous Comprehensions Version: $Revision$ Last-… [View More]Modified: $Date$ Author: Yury Selivanov <yury(a)magic.io> Discussions-To: <python-dev(a)python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 03-Sep-2016 Python-Version: 3.6 Post-History: 03-Sep-2016 Abstract ======== PEP 492 and PEP 525 introduce support for native coroutines and asynchronous generators using ``async`` / ``await`` syntax. This PEP proposes to add asynchronous versions of list, set, dict comprehensions and generator expressions. Rationale and Goals =================== Python has extensive support for synchronous comprehensions, allowing to produce lists, dicts, and sets with a simple and concise syntax. We propose implementing similar syntactic constructions for the asynchronous code. To illustrate the readability improvement, consider the following example:: result = [] async for i in aiter(): if i % 2: result.append(i) With the proposed asynchronous comprehensions syntax, the above code becomes as short as:: result = [i async for i in aiter() if i % 2] The PEP also makes it possible to use the ``await`` expressions in all kinds of comprehensions:: result = [await fun() for fun in funcs] Specification ============= Asynchronous Comprehensions --------------------------- We propose to allow using ``async for`` inside list, set and dict comprehensions. Pending PEP 525 approval, we can also allow creation of asynchronous generator expressions. Examples: * set comprehension: ``{i async for i in agen()}``; * list comprehension: ``[i async for i in agen()]``; * dict comprehension: ``{i: i ** 2 async for i in agen()}``; * generator expression: ``(i ** 2 async for i in agen())``. It is allowed to use ``async for`` along with ``if`` and ``for`` clauses in asynchronous comprehensions and generator expressions:: dataset = {data for line in aiter() async for data in line if check(data)} Asynchronous comprehensions are only allowed inside an ``async def`` function. In principle, asynchronous generator expressions are allowed in any context. However, in Python 3.6, due to ``async`` and ``await`` soft-keyword status, asynchronous generator expressions are only allowed in an ``async def`` function. Once ``async`` and ``await`` become reserved keywords in Python 3.7 this restriction will be removed. ``await`` in Comprehensions --------------------------- We propose to allow the use of ``await`` expressions in both asynchronous and synchronous comprehensions:: result = [await fun() for fun in funcs] result = {await fun() for fun in funcs} result = {fun: await fun() for fun in funcs} result = [await fun() async for fun in funcs] result = {await fun() async for fun in funcs} result = {fun: await fun() async for fun in funcs} This is only valid in ``async def`` function body. Grammar Updates --------------- The proposal requires one change on the grammar level: adding the optional "async" keyword to ``comp_for``:: comp_for: [ASYNC] 'for' exprlist 'in' or_test [comp_iter] The ``comprehension`` AST node will have the new ``is_async`` argument. Backwards Compatibility ----------------------- The proposal is fully backwards compatible. Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: [View Less]

12 31

Atomic counter / atomic increment
by Ben Hoyt Sept. 7, 2016

Sept. 7, 2016

I had to implement a simple atomic counter the other day to count the total number of requests processed in a multi-threaded Python web server. I was doing a demo of "how cool Python is" to my colleagues, and they were generally wowed, but one of the things that made them do a double-take (coming mostly from Scala/Java) was that there was no atomic counter in the standard library. The fact that I and many other folks have implemented such things makes me wonder if it should be in the standard … [View More]

4 5

Re: [Python-ideas] Shuffled
by Arek Bulski Sept. 6, 2016

Sept. 6, 2016

shuffled() should be in the random module, of course. I dont suggest a builtin. Although now that you mentioned it, I could go for that too. There are usage cases where its heavily used, in randomized testing for example. I am sure that there are also other domains where randomization of lists is used. Another reason to put it there is that using shuffle is inconvenient. The fact that I CAN write it myself doesnt mean that it doesnt belong in the standard library. Implementing this in pure … [View More]

4 4

Re: [Python-ideas] shuffled
by Arek Bulski Sept. 6, 2016

Sept. 6, 2016

They closed it. Will they accept a patch if I send it? Where is the repo for the code? Never submitted anything to py code base. -- Arkadiusz Bulski --

3 2

Shuffled
by Arek Bulski Sept. 6, 2016

Sept. 6, 2016

We should add shuffled() analog to sorted. Also shuffle() should return self so mutating methods could be chained. -- Arkadiusz Bulski --

4 3

Semantics for type checking (was: What should a good type checker do?)
by Koos Zevenhoven Sept. 4, 2016

Sept. 4, 2016

On Friday, while replying to a post on python-dev about PEP 526 (variable annotations), I ended up mentioning things that I think a good type checker should do, which seem to differ from the general understanding. The discussion should continue here, though, because the aim of PEP 526 is not to define type-checking semantics. I'll recap the thoughts in this post, but here are links to the original posts, for reference: [1] https://mail.python.org/pipermail/python-dev/2016-September/146069.html… [View More] [2] https://mail.python.org/pipermail/python-dev/2016-September/146090.html Quoting [1], with added explanations in between: On Fri, Sep 2, 2016 at 1:10 PM, Koos Zevenhoven <k7hoven(a)gmail.com> wrote: [...] > > The more you infer types, the less you check them. It's up to the > programmers to choose the amount of annotation. > [...] > > I'm also a little worried about not being able to reannotate a name. > IOW, this should be allowed: x : int = 1 foo(x) x : float = 0.5 bar(x) It turns out that disallowing this was not the PEP's intention, because type-checking semantics is up to the type checker. This has now been clarified in the PEP. Good. [...] >> def eggs(cond:bool): >> x: Optional[List[int]] >> if cond: >> x = None # Now legal >> else: >> x: = [] >> spam(x) >> The above is an example from Mark Shannon's email, and below is my response: > > A good checker should be able to infer that x is a union type at the > point that it's passed to spam, even without the type annotation. For > example: > > def eggs(cond:bool): > if cond: > x = 1 > else: > x = 1.5 > spam(x) # a good type checker infers that x is of type Union[int, float] > Here, there are two inferred types for x, but in different branches of the if statement. So x, which is dynamically typed at runtime, becomes a Union at "static-check time". > Or with annotations: > > def eggs(cond:bool): > if cond: > x : int = foo() # foo may not have a return type hint > else: > x : float = bar() # bar may not have a return type hint > spam(x) # a good type checker infers that x is of type Union[int, float] > > Two explicit annotations, and again neither of them "wins". The inferred type after the if statement is therefore a Union. Below, I'm quoting [2], my response to an email that disagreed with me about the above being something "a good type checker" would do: On Fri, Sep 2, 2016 at 6:49 PM, Koos Zevenhoven <k7hoven(a)gmail.com> wrote: >>> A good checker should be able to infer that x is a union type at the >>> point that it's passed to spam, even without the type annotation. For >>> example: >>> >>> def eggs(cond:bool): >>> if cond: >>> x = 1 >>> else: >>> x = 1.5 >>> spam(x) # a good type checker infers that x is of type Union[int, float] >> >> Oh I really hope not. I wouldn't call that a *good* type checker. I >> would call that a type checker that is overly permissive. > > I guess it's perfectly fine if we disagree about type checking ideals, > and I can imagine the justification for you thinking that way. There > can also be different type checkers, and which can have different > modes. > > But assume (a) that the above function is perfectly working code, and > spam(...) accepts Union[int, float]. Why would I want the type checker > to complain? > > Then again, (b) instead of that being working code, it might be an > error and spam only takes float. No problem, the type checker will > catch that. > > In case of (b), to get the behavior you want (but in my hypothetical > semantics), this could be annotated as > > def eggs(cond:bool): > x : float > if cond: > x = 1 # type checker says error > else: > x = 1.5 > spam(x) > > So here the programmer thinks the type of x should be more constrained > than what spam(...) accepts. The reasoning here is that x is explicitly annotated without an assignment, so the inferred types subsequently assigned to it (int and float) must be compatible with the annotation. Therefore "the good type checker" would say assigning 1 is an error. > > Or you might have something like this > > def eggs(cond:bool): > if cond: > x = 1 > else: > x = 1.5 > # type checker has inferred x to be Union[int, float] > x : float # type checker finds an error > spam(x) > > Here, the same error is found, but at a different location. > Here, x is explicitly annotated as float while its value remains a Union[int, float], as inferred by "the good type checker". Therefore this is an error. One could consider this a type assertation (at static-check time). "The good type checker" should also be able to tell the programmer that it's "x = 1" that made the subsequent "type assertion" fail. In addition to being type *hints*, the annotations can also be considered as "check points". -- Koos -- + Koos Zevenhoven + http://twitter.com/k7hoven + [View Less]

3 6

Re: [Python-ideas] [Python-Dev] What should a good type checker do? (was: Please reject or postpone PEP 526)
by Stephen J. Turnbull Sept. 4, 2016

Sept. 4, 2016

Please respect Reply-To, set to python-ideas. Greg Ewing writes: > Chris Angelico wrote: > > Forcing people to write 1.0 just to be compatible with 1.5 will cause > > a lot of annoyance. > > Indeed, this would be unacceptable IMO. But "forcing" won't happen. Just ignore the warning. *All* such Python programs will continue to run (or crash) exactly as if the type declarations weren't there. If you don't like the warning, either don't run the typechecker, or … [View More]

3 4

What should a good typechecker do?
by Stephen J. Turnbull Sept. 3, 2016

Sept. 3, 2016

Guido van Rossum writes: > But that's not what type comments mean! They don't annotate the > expression. They annotate the variable. In PEP 484. But syntactically, AFAICS in an initialization that's a distinction without a difference. It would be perfectly possible to write a checker that allows if cond: x: str = "a string, what else?" else: x: int = 1 and infers the union, and even infers the types from the expressions in if cond: x = "a … [View More]

2 1

Extending expressions using ellipsis
by Shane Hathaway Sept. 1, 2016

Sept. 1, 2016

Hi, I write a lot of SQLAlchemy code that looks more or less like this: rows = ( dbsession.query(Table1) .join( Table2, Table2.y = Table1.y) .filter(Table1.x = xx) .all()) The expressions get very long and nearly always need to be spread to multiple lines. I've tried various styles and have chosen the style above as the most tasteful available. Pros of the existing syntax: - It's possible to indent clearly and consistently. - Nested indentation works out OK. -… [View More]

12 18

Improve the error message for impossible multiple inheritance
by Neil Girdhar Sept. 1, 2016

Sept. 1, 2016

I sometimes get the error message: TypeError: Cannot create a consistent method resolution order (MRO) for bases … I suggest that Python report * cycles (if there are any), or * a list of orderings of direct base classes (if there are no cycles). A cycle for the base classes A, B, C, D can be reported as follows: TypeError: Cannot create a consistent method resolution order (MRO) because A inherits from B (through X, Y), but B inherits from A (through Z). The topological sort could be … [View More]

1 0

real numbers with SI scale factors: next steps
by Ken Kundert Sept. 1, 2016

Sept. 1, 2016

Okay, let's try to wrap this up. In summary I proposed three things: 1. A change to the Python lexer to accept SI literal as an alternative, but not replacement to, E-notation. As an optional feature, simple units could be added to the end but would be largely ignored. So the following would be accepted: freq = 2.4GHz r = 1k l = 10nm The idea in accepting units was to allow them to be specified when convenient as additional documentation on the meaning of … [View More]the number. Objections: a. Acceptance of the abbreviation for Exa (E) overlaps with E-notation (1E+1 could represent 1e18 + 1 or 10). A suggestion to change the prefix from E to X conflicts with a proposal to use X, W, and V to represent 10^27, 10^30, and 10^33 (en.wikipedia.org/wiki/Metric_prefix) b. Allowing the units to be specified will lead some users to assume a dimensional analysis is being performed when in fact the units are ignored. This false sense of security could lead to bugs. c. The proposal only supports simple units, not compound units such as m/s. So even if hooks were provided to allow access to the units to support an add-on dimensional analysis capability, an additional mechanism would have to be provided to support compound units. d. Many people objected to allowing the use of naked scale factors as a perversion of the standard. 2. A change to the float() function so that it accepts SI scale factors and units. This extension naturally follows from the first: the float function should accept anything the Python parser accepts. For example: freq = float('2.4GHz') r = float('1k') l = float('10nm') Objections: a. The Exa objection from the above proposal is problematic here as well. b. Things that used to be errors are now no longer errors. This could cause problems if a program was counting on float('1k') to be an error. 3. A change to the various string formatting mechanisms to allow outputting real numbers with SI scale factors: >>> print('Speed of light in a vacuum: {:r}m/s.'.format(2.9979e+08)) Speed of light in a vacuum: 299.79 Mm/s. >>> print('Speed of sound in water: %rm/s.' % 1481 Speed of sound in water: 1.481 km/s. Objections: No objections were raised that I recall, however here is something else to consider: a. Should we also provide mechanism for the binary scale factors (Ki, Mi, ..., Yi)? For example: '{:b}B'.format(2**30) --> 1 GiB. On proposed extension 1 (native support for SI literals) my conclusion is that we did not reach any sense of consensus and there was considerable opposition to my proposal. There was much less discussion on extensions 2 & 3, so it is hard to say whether consensus was reached. So, given all this, I would like to make the following recommendations: 1. No action should be taken. 2. The main justification to modifying float() was to make it consistent with the extended Python language. Without extension 1, this justification goes away. However the need to be able to easily convert strings of numbers with SI scale factors into floats still exists. This should be handled by adding a library or extending an existing library. 3. Allowing numbers to be formatted with SI prefixes is useful and not controversial. The 'r' and 'b' format codes should be added to the various string formatting mechanisms. What do you think? -Ken [View Less]

14 28