Mailman 3 February 2018 - Python-ideas

Adding a thin wrapper class around the functions in stdlib.heapq
by bunslow Feb. 18, 2022

Feb. 18, 2022

Nothing so bombastic this time. The heapq functions are basically all named "heapsomething", and basically all take a "heap" for their first argument, with supplementary args coming after. It's a textbook example of the (hypothetical) Object Oriented Manifesto™ where defining a class increases type safety and programmers' conceptual clarity. There're practically no drawbacks, and the code to be added would be very simple. Updating the tests and docs would probably be harder. In pure Python, … [View More]

17 32

Dataclasses, keyword args, and inheritance
by George Leslie-Waksman March 12, 2021

March 12, 2021

The proposed implementation of dataclasses prevents defining fields with defaults before fields without defaults. This can create limitations on logical grouping of fields and on inheritance. Take, for example, the case: @dataclass class Foo: some_default: dict = field(default_factory=dict) @dataclass class Bar(Foo): other_field: int this results in the error: 5 @dataclass ----> 6 class Bar(Foo): 7 other_field: int 8 ~/.pyenv/versions/3.6.2/envs/… [View More]clover_pipeline/lib/python3.6/site-packages/dataclasses.py in dataclass(_cls, init, repr, eq, order, hash, frozen) 751 752 # We're called as @dataclass, with a class. --> 753 return wrap(_cls) 754 755 ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in wrap(cls) 743 744 def wrap(cls): --> 745 return _process_class(cls, repr, eq, order, hash, init, frozen) 746 747 # See if we're being called as @dataclass or @dataclass(). ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in _process_class(cls, repr, eq, order, hash, init, frozen) 675 # in __init__. Use "self" if possible. 676 '__dataclass_self__' if 'self' in fields --> 677 else 'self', 678 )) 679 if repr: ~/.pyenv/versions/3.6.2/envs/clover_pipeline/lib/python3.6/site-packages/dataclasses.py in _init_fn(fields, frozen, has_post_init, self_name) 422 seen_default = True 423 elif seen_default: --> 424 raise TypeError(f'non-default argument {f.name!r} ' 425 'follows default argument') 426 TypeError: non-default argument 'other_field' follows default argument I understand that this is a limitation of positional arguments because the effective __init__ signature is: def __init__(self, some_default: dict = <something>, other_field: int): However, keyword only arguments allow an entirely reasonable solution to this problem: def __init__(self, *, some_default: dict = <something>, other_field: int): And have the added benefit of making the fields in the __init__ call entirely explicit. So, I propose the addition of a keyword_only flag to the @dataclass decorator that renders the __init__ method using keyword only arguments: @dataclass(keyword_only=True) class Bar(Foo): other_field: int --George Leslie-Waksman [View Less]

12 35

@classproperty, @abc.abstractclasspropery, etc.
by K. Richard Pixley Dec. 16, 2020

Dec. 16, 2020

There's a whole matrix of these and I'm wondering why the matrix is currently sparse rather than implementing them all. Or rather, why we can't stack them as: class foo(object): @classmethod @property def bar(cls, ...): ... Essentially the permutation are, I think: {'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable attribute}. concreteness implicit first arg type name comments {unadorned} {unadorned} method def foo(): exists … [View More]now {unadorned} {unadorned} property @property exists now {unadorned} {unadorned} non-callable attribute x = 2 exists now {unadorned} static method @staticmethod exists now {unadorned} static property @staticproperty proposing {unadorned} static non-callable attribute {degenerate case - variables don't have arguments} unnecessary {unadorned} class method @classmethod exists now {unadorned} class property @classproperty or @classmethod;@property proposing {unadorned} class non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract {unadorned} method @abc.abstractmethod exists now abc.abstract {unadorned} property @abc.abstractproperty exists now abc.abstract {unadorned} non-callable attribute @abc.abstractattribute or @abc.abstract;@attribute proposing abc.abstract static method @abc.abstractstaticmethod exists now abc.abstract static property @abc.staticproperty proposing abc.abstract static non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract class method @abc.abstractclassmethod exists now abc.abstract class property @abc.abstractclassproperty proposing abc.abstract class non-callable attribute {degenerate case - variables don't have arguments} unnecessary I think the meanings of the new ones are pretty straightforward, but in case they are not... @staticproperty - like @property only without an implicit first argument. Allows the property to be called directly from the class without requiring a throw-away instance. @classproperty - like @property, only the implicit first argument to the method is the class. Allows the property to be called directly from the class without requiring a throw-away instance. @abc.abstractattribute - a simple, non-callable variable that must be overridden in subclasses @abc.abstractstaticproperty - like @abc.abstractproperty only for @staticproperty @abc.abstractclassproperty - like @abc.abstractproperty only for @classproperty --rich [View Less]

10 15

Re: [Python-ideas] Allow star unpacking within an slice expression
by Neil Girdhar March 7, 2020

March 7, 2020

I didn't think of this when we were discussing 448. I ran into this today, so I agree with you that it would be nice to have this. Best, Neil On Monday, December 4, 2017 at 1:02:09 AM UTC-5, Eric Wieser wrote: > > Hi, > > I've been thinking about the * unpacking operator while writing some numpy > code. PEP 448 allows the following: > > values = 1, *some_tuple, 2 > object[(1, *some_tuple, 2)] > > It seems reasonable to me that it should be extended to … [View More]

2 2

Specify number of items to allocate for array.array() constructor
by Sven Rahmann Feb. 22, 2020

Feb. 22, 2020

At the moment, the array module of the standard library allows to create arrays of different numeric types and to initialize them from an iterable (eg, another array). What's missing is the possiblity to specify the final size of the array (number of items), especially for large arrays. I'm thinking of suffix arrays (a text indexing data structure) for large texts, eg the human genome and its reverse complement (about 6 billion characters from the alphabet ACGT). The suffix array is a long int … [View More]

14 20

Please consider adding of functions file system operations to pathlib
by George Fischhof Nov. 16, 2019

Nov. 16, 2019

Good day all, as a continuation of thread "OS related file operations (copy, move, delete, rename...) should be placed into one module" https://mail.python.org/pipermail/python-ideas/2017-January/044217.html please consider making pathlib to a central file system module with putting file operations (copy, move, delete, rmtree etc) into pathlib. BR, George

3 3

Asynchronous exception handling around with/try statement borders
by Erik Bray Sept. 24, 2018

Sept. 24, 2018

Hi folks, I normally wouldn't bring something like this up here, except I think that there is possibility of something to be done--a language documentation clarification if nothing else, though possibly an actual code change as well. I've been having an argument with a colleague over the last couple days over the proper way order of statements when setting up a try/finally to perform cleanup of some action. On some level we're both being stubborn I think, and I'm not looking for resolution … [View More]as to who's right/wrong or I wouldn't bring it to this list in the first place. The original argument was over setting and later restoring os.environ, but we ended up arguing over threading.Lock.acquire/release which I think is a more interesting example of the problem, and he did raise a good point that I do want to bring up. </prologue> My colleague's contention is that given lock = threading.Lock() this is simply *wrong*: lock.acquire() try: do_something() finally: lock.release() whereas this is okay: with lock: do_something() Ignoring other details of how threading.Lock is actually implemented, assuming that Lock.__enter__ calls acquire() and Lock.__exit__ calls release() then as far as I've known ever since Python 2.5 first came out these two examples are semantically *equivalent*, and I can't find any way of reading PEP 343 or the Python language reference that would suggest otherwise. However, there *is* a difference, and has to do with how signals are handled, particularly w.r.t. context managers implemented in C (hence we are talking CPython specifically): If Lock.__enter__ is a pure Python method (even if it maybe calls some C methods), and a SIGINT is handled during execution of that method, then in almost all cases a KeyboardInterrupt exception will be raised from within Lock.__enter__--this means the suite under the with: statement is never evaluated, and Lock.__exit__ is never called. You can be fairly sure the KeyboardInterrupt will be raised from somewhere within a pure Python Lock.__enter__ because there will usually be at least one remaining opcode to be evaluated, such as RETURN_VALUE. Because of how delayed execution of signal handlers is implemented in the pyeval main loop, this means the signal handler for SIGINT will be called *before* RETURN_VALUE, resulting in the KeyboardInterrupt exception being raised. Standard stuff. However, if Lock.__enter__ is a PyCFunction things are quite different. If you look at how the SETUP_WITH opcode is implemented, it first calls the __enter__ method with _PyObjet_CallNoArg. If this returns NULL (i.e. an exception occurred in __enter__) then "goto error" is executed and the exception is raised. However if it returns non-NULL the finally block is set up with PyFrame_BlockSetup and execution proceeds to the next opcode. At this point a potentially waiting SIGINT is handled, resulting in KeyboardInterrupt being raised while inside the with statement's suite, and finally block, and hence Lock.__exit__ are entered. Long story short, because Lock.__enter__ is a C function, assuming that it succeeds normally then with lock: do_something() always guarantees that Lock.__exit__ will be called if a SIGINT was handled inside Lock.__enter__, whereas with lock.acquire() try: ... finally: lock.release() there is at last a small possibility that the SIGINT handler is called after the CALL_FUNCTION op but before the try/finally block is entered (e.g. before executing POP_TOP or SETUP_FINALLY). So the end result is that the lock is held and never released after the KeyboardInterrupt (whether or not it's handled somehow). Whereas, again, if Lock.__enter__ is a pure Python function there's less likely to be any difference (though I don't think the possibility can be ruled out entirely). At the very least I think this quirk of CPython should be mentioned somewhere (since in all other cases the semantic meaning of the "with:" statement is clear). However, I think it might be possible to gain more consistency between these cases if pending signals are checked/handled after any direct call to PyCFunction from within the ceval loop. Sorry for the tl;dr; any thoughts? [View Less]

7 15

Positional-only parameters
by Victor Stinner Sept. 11, 2018

Sept. 11, 2018

Hi, For technical reasons, many functions of the Python standard libraries implemented in C have positional-only parameters. Example: ------- $ ./python Python 3.7.0a0 (default, Feb 25 2017, 04:30:32) >>> help(str.replace) replace(self, old, new, count=-1, /) # <== notice "/" at the end ... >>> "a".replace("x", "y") # ok 'a' >>> "a".replace(old="x", new="y") # ERR! TypeError: replace() takes at least 2 arguments (0 given) ------- When converting the … [View More]

19 27

Implicit string literal concatenation considered harmful?
by Guido van Rossum March 14, 2018

March 14, 2018

I just spent a few minutes staring at a bug caused by a missing comma -- I got a mysterious argument count error because instead of foo('a', 'b') I had written foo('a' 'b'). This is a fairly common mistake, and IIRC at Google we even had a lint rule against this (there was also a Python dialect used for some specific purpose where this was explicitly forbidden). Now, with modern compiler technology, we can (and in fact do) evaluate compile-time string literal concatenation with the '+' … [View More]

51 165

PEP 572: Statement-Local Name Bindings
by Chris Angelico March 3, 2018

March 3, 2018

This is a suggestion that comes up periodically here or on python-dev. This proposal introduces a way to bind a temporary name to the value of an expression, which can then be used elsewhere in the current statement. The nicely-rendered version will be visible here shortly: https://www.python.org/dev/peps/pep-0572/ ChrisA PEP: 572 Title: Syntax for Statement-Local Name Bindings Author: Chris Angelico <rosuav(a)gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst … [View More]Created: 28-Feb-2018 Python-Version: 3.8 Post-History: 28-Feb-2018 Abstract ======== Programming is all about reusing code rather than duplicating it. When an expression needs to be used twice in quick succession but never again, it is convenient to assign it to a temporary name with very small scope. By permitting name bindings to exist within a single statement only, we make this both convenient and safe against collisions. Rationale ========= When an expression is used multiple times in a list comprehension, there are currently several suboptimal ways to spell this, and no truly good ways. A statement-local name allows any expression to be temporarily captured and then used multiple times. Syntax and semantics ==================== In any context where arbitrary Python expressions can be used, a named expression can appear. This must be parenthesized for clarity, and is of the form `(expr as NAME)` where `expr` is any valid Python expression, and `NAME` is a simple name. The value of such a named expression is the same as the incorporated expression, with the additional side-effect that NAME is bound to that value for the remainder of the current statement. Just as function-local names shadow global names for the scope of the function, statement-local names shadow other names for that statement. They can also shadow each other, though actually doing this should be strongly discouraged in style guides. Example usage ============= These list comprehensions are all approximately equivalent:: # Calling the function twice stuff = [[f(x), f(x)] for x in range(5)] # Helper function def pair(value): return [value, value] stuff = [pair(f(x)) for x in range(5)] # Inline helper function stuff = [(lambda v: [v,v])(f(x)) for x in range(5)] # Extra 'for' loop - see also Serhiy's optimization stuff = [[y, y] for x in range(5) for y in [f(x)]] # Expanding the comprehension into a loop stuff = [] for x in range(5): y = f(x) stuff.append([y, y]) # Using a statement-local name stuff = [[(f(x) as y), y] for x in range(5)] If calling `f(x)` is expensive or has side effects, the clean operation of the list comprehension gets muddled. Using a short-duration name binding retains the simplicity; while the extra `for` loop does achieve this, it does so at the cost of dividing the expression visually, putting the named part at the end of the comprehension instead of the beginning. Statement-local name bindings can be used in any context, but should be avoided where regular assignment can be used, just as `lambda` should be avoided when `def` is an option. Open questions ============== 1. What happens if the name has already been used? `(x, (1 as x), x)` Currently, prior usage functions as if the named expression did not exist (following the usual lookup rules); the new name binding will shadow the other name from the point where it is evaluated until the end of the statement. Is this acceptable? Should it raise a syntax error or warning? 2. The current implementation [1] implements statement-local names using a special (and mostly-invisible) name mangling. This works perfectly inside functions (including list comprehensions), but not at top level. Is this a serious limitation? Is it confusing? 3. The interaction with locals() is currently[1] slightly buggy. Should statement-local names appear in locals() while they are active (and shadow any other names from the same function), or should they simply not appear? 4. Syntactic confusion in `except` statements. While technically unambiguous, it is potentially confusing to humans. In Python 3.7, parenthesizing `except (Exception as e):` is illegal, and there is no reason to capture the exception type (as opposed to the exception instance, as is done by the regular syntax). Should this be made outright illegal, to prevent confusion? Can it be left to linters? 5. Similar confusion in `with` statements, with the difference that there is good reason to capture the result of an expression, and it is also very common for `__enter__` methods to return `self`. In many cases, `with expr as name:` will do the same thing as `with (expr as name):`, adding to the confusion. References ========== .. [1] Proof of concept / reference implementation (https://github.com/Rosuav/cpython/tree/statement-local-variables) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: [View Less]

19 88

Medium for discussion potential changes to python (was: PEP 572: Statement-Local Name Bindings)
by Alex Walters March 2, 2018

March 2, 2018

That should probably be its own thread From: Python-ideas [mailto:python-ideas-bounces+tritium-list=sdamon.com@python.org] On Behalf Of Robert Vanden Eynde Sent: Wednesday, February 28, 2018 4:48 PM Cc: python-ideas <python-ideas(a)python.org> Subject: Re: [Python-ideas] PEP 572: Statement-Local Name Bindings We are currently like a dozen of people talking about multiple sections of a single subject. Isn't it easier to talk on a forum? Am I the only one who thinks mailing list … [View More]isn't easy when lots of people talking about multiple subjects? Of course we would put the link in the mailing list so that everyone can join. A forum (or just few "issues" thread on github) is where we could have different thread in parallel, in my messages I end up with like 10 comments not all related, in a forum we could talk about everything and it would still be organized by subjects. Also, it's more interactive than email on a global list, people can talk to each other in parallel, if I want to answer about a mail that was 10 mail ago, it gets quickly messy. We could all discuss on a gist or some "Issues" thread on GitHub. 2018-02-28 22:38 GMT+01:00 Robert Vanden Eynde <robertve92(a)gmail.com <mailto:robertve92@gmail.com> >: Le 28 févr. 2018 11:43, "Chris Angelico" <rosuav(a)gmail.com <mailto:rosuav@gmail.com> > a écrit : > It's still right-to-left, which is as bad as middle-outward once you > combine it with normal left-to-right evaluation. Python has very > little of this [..] I agree [....] >> 2) talking about the implementation of thektulu in the "where =" part. > ? In the Alternate Syntax, I was talking about adding a link to the thektulu (branch where-expr) <https://github.com/thektulu/cpython/commits/where-expr> implementation as a basis of proof of concept (as you did with the other syntax). >> 3) "C problem that an equals sign in an expression can now create a name inding, rather than performing a comparison." As you agreed, with the "ch with ch = getch()" syntax we won't accidentally switch a "==" for a "=". I agree this syntax : ``` while (ch with ch = getch()): ... ``` doesn't read very well, but in the same way as in C or Java while(ch = getch()){} or worse ((ch = getch()) != null) syntax. Your syntax "while (getch() as ch):" may have a less words, but is still not clearer. As we spoke on Github, having this syntax in a while is only useful if the variable does leak. >> 5) Any expression vs "post for" only > I don't know what the benefit is here, but sure. As long as the > grammar is unambiguous, I don't see any particular reason to reject > this. I would like to see a discussion of pros and cons, some might think like me or disagree, that's a strong langage question. > 6) with your syntax, how does the simple case work (y+2 with y = x+1) ? What simple case? The case where you only use the variable once? I'd write it like this: (x + 1) + 2 >> The issue is not only about reusing variable. > If you aren't using the variable multiple times, there's no point > giving it a name. Unless I'm missing something here? Yes, variables are not there "just because we reuse them", but also to include temporary variables to better understand the code. Same for functions, you could inline functions when used only once, but you introduce them for clarity no ? ``` a = v ** 2 / R # the acceleration in a circular motion f = m * a # law of Newton ``` could be written as ``` f = m * (v ** 2 / R) # compute the force, trivial ``` But having temporary variables help a lot to understand the code, otherwise why would we create temporary variables ? I can give you an example where you do a process and each time the variable is used only one. >> 8) >> (lambda y: [y, y])(x+1) >> Vs >> (lambda y: [y, y])(y=x+1) Ewww. Remind me what the benefit is of writing the variable name that many times? "Explicit" doesn't mean "utterly verbose". Yep it's verbose, lambdas are verbose, that's why we created this PEP isn't it :) > 10) Chaining, in the case of the "with =", in thektulu, parenthesis were > mandatory: > > print((z+3 with z = y+2) with y = x+2) > > What happens when the parenthesis are dropped ? > > print(z+3 with y = x+2 with z = y+2) > > Vs > > print(z+3 with y = x+2 with z = y+2) > > I prefer the first one be cause it's in the same order as the "post for" > > [z + 3 for y in [ x+2 ] for z in [ y+2 ]] > With my proposal, the parens are simply mandatory. Extending this to > make them optional can come later. Indeed, but that's still questions that can be asked. >> 11) Scoping, in the case of the "with =" syntax, I think the parenthesis >> introduce a scope : >> >> print(y + (y+1 where y = 2)) >> >> Would raise a SyntaxError, it's probably better for the variable beeing >> local and not in the current function (that would be a mess). >> >> Remember that in list comp, the variable is not leaked : >> >> x = 5 >> stuff = [y+2 for y in [x+1] >> print(y) # SyntaxError > Scoping is a fundamental part of both my proposal and the others I've > seen here. (BTW, that would be a NameError, not a SyntaxError; it's > perfectly legal to ask for the name 'y', it just hasn't been given any > value.) By my definition, the variable is locked to the statement that > created it, even if that's a compound statement. By the definition of > a "(expr given var = expr)" proposal, it would be locked to that > single expression. Confer the discussion on scoping on github (https://github.com/python/peps/commit/2b4ca20963a24cf5faac054226857ea970547…) : """ In the current implementation it looks like it is like a regular assignment (function local then). Therefore in the expression usage, the usefulness would be debatable (just assign before). But in a list comprehension after the for (as I mentioned in my mail), aka. when used as a replacement for for y in [ x + 1 ] this would make sense. But I think that it would be much better to have a local scope, in the parenthesis. So that print(y+2 where y = x + 1) wouldn't leak y. And when there are no parenthesis like in a = y+2 where y = x+1, it would imply one, giving the same effect as a = (y+2 where y = x+1). Moreover, it would naturally shadow variables in the outermost scope. This would imply while data where data = sock.read(): does not leak data but as a comparison with C and Java, the syntax while((data = sock.read()) != null) is really really ugly and confusing. """ [View Less]

5 7

List assignment - extended slicing inconsistency
by Alexander Heger Feb. 24, 2018

Feb. 24, 2018

What little documentation I could find, providing a stride on the assignment target for a list is supposed to trigger 'advanced slicing' causing element-wise replacement - and hence requiring that the source iterable has the appropriate number of elements. >>> a = [0,1,2,3] >>> a[::2] = [4,5] >>> a [4, 1, 5, 3] >>> a[::2] = [4,5,6] Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: attempt to assign sequence of … [View More]size 3 to extended slice of size 2 This is in contrast to regular slicing (*without* a stride), allowing to replace a *range* by another sequence of arbitrary length. >>> a = [0,1,2,3] >>> a[:3] = [4] >>> a [4, 3] Issue ===== When, however, a stride of `1` is specified, advanced slicing is not triggered. >>> a = [0,1,2,3] >>> a[:3:1] = [4] >>> a [4, 3] If advanced slicing had been triggered, there should have been a ValueError instead. Expected behaviour: >>> a = [0,1,2,3] >>> a[:3:1] = [4] Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: attempt to assign sequence of size 1 to extended slice of size 3 I think that is an inconsistency in the language that should be fixed. Why do we need this? ==================== One may want this as extra check as well so that list does not change size. Depending on implementation, it may come with performance benefits as well. One could, though, argue that you still get the same result if you do all correctly >>> a = [0,1,2,3] >>> a[:3:1] = [4,5,6] >>> a [4, 5, 6, 3] But I disagree that there should be no error when it is wrong. *Strides that are not None should always trigger advanced slicing.* Other Data Types ================ This change should also be applied to bytearray, etc., though see below. Concerns ======== It may break some code that uses advanced slicing and expects regular slicing to occur? These cases should be rare, and the error message should be clear enough to allow fixes? I assume these cases should be exceptionally rare. If the implementation relies on `slice.indices(len(seq))[2] == 1` to determine about advance slicing or not, that would require some refactoring. If it is only `slice.stride in (1, None)` then this could easily replaced by checking against None. Will there be issues with syntax consistency with other data types, in particular outside the core library? - I always found that the dynamic behaviour of lists w/r non-advanced slicing to be somewhat peculiar in the first place, though, undeniably, it can be immensely useful. - Most external data types with fixed memory such as numpy do not have this dynamic flexibility, and the behavior of regular slicing on assignment is the same as regular slicing. The proposed change would increase consistency with these other data types. More surprises ============== >>> import array >>> a[1::2] = a[3:3] Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: attempt to assign sequence of size 0 to extended slice of size 2 whereas >>> a = [1,2,3,4,5] >>> a[1::2] = a[3:3] Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: attempt to assign sequence of size 0 to extended slice of size 2 >>> a = bytearray(b'12345') >>> a[1::2] = a[3:3] >>> a bytearray(b'135') but numpy >>> import numpy as np >>> a = np.array([1,2,3,4,5]) >>> a[1::2] = a[3:3] Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: could not broadcast input array from shape (0) into shape (2) and >>> import numpy as np >>> a[1:2] = a[3:3] Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: could not broadcast input array from shape (0) into shape (1) The latter two as expected. memoryview behaves the same. Issue 2 ======= Whereas NumPy is know to behave differently as a data type with fixed memory layout, and is not part of the standard library anyway, the difference in behaviour between lists and arrays I find disconcerting. This should be resolved to a consistent behaviour. Proposal 2 ========== Arrays and bytearrays should should adopt the same advanced slicing behaviour I suggest for lists. Concerns 2 ========== This has the potential for a lot more side effects in existing code, but as before in most cases error message should be triggered. Summary ======= I find it it not acceptable as a good language design that there is a large range of behaviour on slicing in assignment target for the different native (and standard library) data type of seemingly similar kind, and that users have to figure out for each data type by testing - or at the very least remember if documented - how it behaves on slicing in assignment targets. There should be a consistent behaviour at the very least, ideally even one with a clear user interface as suggested for lists. -Alexander [View Less]

6 7

Temporary variables in comprehensions
by fhsxfhsx Feb. 23, 2018

Feb. 23, 2018

As far as I can see, a comprehension like alist = [f(x) for x in range(10)] is better than a for-loop for x in range(10): alist.append(f(x)) because the previous one shows every element of the list explicitly so that we don't need to handle `append` mentally. But when it comes to something like [f(x) + g(f(x)) for x in range(10)] you find you have to sacrifice some readableness if you don't want two f(x) which might slow down your code. Someone may argue that one can write [y + g(y) for y … [View More]

13 24

Boolean ABC similar to what's provided in the 'numbers' module
by Sylvain MARIE Feb. 21, 2018

Feb. 21, 2018

The numbers module provides very useful ABC for the 'numeric tower', able to abstract away the differences between python primitives and for example numpy primitives. I could not find any equivalent for Booleans. However numpy defines np.bool too, so being able to have an abstract Boolean class for both python bool and numpy bool would be great. Here is a version that I included in valid8 in the meantime ----------------------- class Boolean(metaclass=ABCMeta): """ An abstract base … [View More]

12 26

Why CPython is still behind in performance for some widely used patterns ?
by Pau Freixes Feb. 21, 2018

Feb. 21, 2018

Hi, This mail is the consequence of a true story, a story where CPython got defeated by Javascript, Java, C# and Go. One of the teams of the company where Im working had a kind of benchmark to compare the different languages on top of their respective "official" web servers such as Node.js, Aiohttp, Dropwizard and so on. The test by itself was pretty simple and tried to test the happy path of the logic, a piece of code that fetches N rules from another system and then apply them to X … [View More]

22 40

importlib: making FileFinder easier to extend
by Erik Bray Feb. 20, 2018

Feb. 20, 2018

Hello, Brief problem statement: Let's say I have a custom file type (say, with extension .foo) and these .foo files are included in a package (along with other Python modules with standard extensions like .py and .so), and I want to make these .foo files importable like any other module. On its face, importlib.machinery.FileFinder makes this easy. I make a loader for my custom file type (say, FooSourceLoader), and I can use the FileFinder.path_hook helper like: sys.path_hooks.insert(0, … [View More]FileFinder.path_hook((FooSourceLoader, ['.foo']))) sys.path_importer_cache.clear() Great--now I can import my .foo modules like any other Python module. However, any standard Python modules now cannot be imported. The way PathFinder sys.meta_path hook works, sys.path_hooks entries are first-come-first-serve, and furthermore FileFinder.path_hook is very promiscuous--it will take over module loading for *any* directory on sys.path, regardless what the file extensions are in that directory. So although this mechanism is provided by the stdlib, it can't really be used for this purpose without breaking imports of normal modules (and maybe it's not intended for that purpose, but the documentation is unclear). There are a number of different ways one could get around this. One might be to pass FileFinder.path_hook loaders/extension pairs for all the basic file types known by the Python interpreter. Unfortunately there's no great way to get that information. *I* know that I want to support .py, .pyc, .so etc. files, and I know which loaders to use for them. But that's really information that should belong to the Python interpreter, and not something that should be reverse-engineered. In fact, there is such a mapping provided by importlib.machinery._get_supported_file_loaders(), but this is not a publicly documented function. One could probably think of other workarounds. For example you could implement a custom sys.meta_path hook. But I think it shouldn't be necessary to go to higher levels of abstraction in order to do this--the default sys.path handler should be able to handle this use case. In order to support adding support for new file types to sys.path_hooks, I ended up implementing the following hack: ############################################################# import os import sys from importlib.abc import PathEntryFinder @PathEntryFinder.register class MetaFileFinder: """ A 'middleware', if you will, between the PathFinder sys.meta_path hook, and sys.path_hooks hooks--particularly FileFinder. The hook returned by FileFinder.path_hook is rather 'promiscuous' in that it will handle *any* directory. So if one wants to insert another FileFinder.path_hook into sys.path_hooks, that will totally take over importing for any directory, and previous path hooks will be ignored. This class provides its own sys.path_hooks hook as follows: If inserted on sys.path_hooks (it should be inserted early so that it can supersede anything else). Its find_spec method then calls each hook on sys.path_hooks after itself and, for each hook that can handle the given sys.path entry, it calls the hook to create a finder, and calls that finder's find_spec. So each sys.path_hooks entry is tried until a spec is found or all finders are exhausted. """ def __init__(self, path): if not os.path.isdir(path): raise ImportError('only directories are supported', path=path) self.path = path self._finder_cache = {} def __repr__(self): return '{}({!r})'.format(self.__class__.__name__, self.path) def find_spec(self, fullname, target=None): if not sys.path_hooks: return None for hook in sys.path_hooks: if hook is self.__class__: continue finder = None try: if hook in self._finder_cache: finder = self._finder_cache[hook] if finder is None: # We've tried this finder before and got an ImportError continue except TypeError: # The hook is unhashable pass if finder is None: try: finder = hook(self.path) except ImportError: pass try: self._finder_cache[hook] = finder except TypeError: # The hook is unhashable for some reason so we don't bother # caching it pass if finder is not None: spec = finder.find_spec(fullname, target) if spec is not None: return spec # Module spec not found through any of the finders return None def invalidate_caches(self): for finder in self._finder_cache.values(): finder.invalidate_caches() @classmethod def install(cls): sys.path_hooks.insert(0, cls) sys.path_importer_cache.clear() ############################################################# This works, for example, like: >>> MetaFileFinder.install() >>> sys.path_hooks.append(FileFinder.path_hook((SourceFileLoader, ['.foo']))) And now, .foo modules are importable, without breaking support for the built-in module types. This is still overkill though. I feel like there should instead be a way to, say, extend a sys.path_hooks hook based on FileFinder so as to be able to support loading other file types, without having to go above the default sys.meta_path hooks. A small, but related problem I noticed in the way FileFinder.path_hook is implemented, is that for almost *every directory* that gets cached in sys.path_importer_cache, a new FileFinder instance is created with its own self._loaders attribute, each containing a copy of the same list of (loader, extensions) tuples. I calculated that on one large project this alone accounted for nearly 1 MB. Not a big deal in the grand scheme of things, but still a bit overkill. ISTM it would kill two birds with one stone if FileFinder were changed, or there were a subclass thereof, that had a class attribute containing the standard loader/extension mappings. This in turn could simply be appended to in order to support new extension types. Thanks, E [View Less]

2 1

PEP 468
by guido Feb. 19, 2018

Feb. 19, 2018

Gesendet von Mail für Windows 10

2 1

Coming up with an alternative to PEP 505's None-aware operators
by Nick Coghlan Feb. 19, 2018

Feb. 19, 2018

The recent thread on variable assignment in comprehensions has prompted me to finally share https://gist.github.com/ncoghlan/a1b0482fc1ee3c3a11fc7ae64833a315 with a wider audience (see the comments there for some notes on iterations I've already been through on the idea). == The general idea == The general idea would be to introduce a *single* statement local reference using a new keyword with a symbolic prefix: "?it" * `(?it=expr)` is a new atomic expression for an "it reference binding" (… [View More]whitespace would be permitted around "?it" and "=", but PEP 8 would recommend against it in general) * subsequent subexpressions (in execution order) can reference the bound subexpression using `?it` (an "it reference") * `?it` is reset between statements, including before entering the suite within a compound statement (if you want a persistent binding, use a named variable) * for conditional expressions, put the reference binding in the conditional, as that gets executed first * to avoid ambiguity, especially in function calls (where it could be confused with keyword argument syntax), the parentheses around reference bindings are always required * unlike regular variables, you can't close over statement local references (the nested scope will get an UnboundLocalError if you try it) The core inspiration here is English pronouns (hence the choice of keyword): we don't generally define arbitrary terms in the middle of sentences, but we *do* use pronouns to refer back to concepts introduced earlier in the sentence. And while it's not an especially common practice, pronouns are sometimes even used in a sentence *before* the concept they refer to ;) If we did pursue this, then PEPs 505, 532, and 535 would all be withdrawn or rejected (with the direction being to use an it-reference instead). == Examples == `None`-aware attribute access: value = ?it.strip()[4:].upper() if (?it=var1) is not None else None `None`-aware subscript access: value = ?it[4:].upper() if (?it=var1) is not None else None `None`-coalescense: value = ?it if (?it=var1) is not None else ?it if (?it=var2) is not None else var3 `NaN`-coalescence: value = ?it if not math.isnan((?it=var1)) else ?it if not math.isnan((?that=var2)) else var3 Conditional function call: value = ?it() if (?it=calculate) is not None else default Avoiding repeated evaluation of a comprehension filter condition: filtered_values = [?it for x in keys if (?it=get_value(x)) is not None] Avoiding repeated evaluation for range and slice bounds: range((?it=calculate_start()), ?it+10) data[(?it=calculate_start()):?it+10] Avoiding repeated evaluation in chained comparisons: value if (?it=lower_bound()) <= value < ?it+tolerance else 0 Avoiding repeated evaluation in an f-string: print(f"{?it=get_value()!r} is printed in pure ASCII as {?it!a} and in Unicode as {?it}" == Possible future extensions == One possible future extension would be to pursue PEP 3150, treating the nested namespace as an it reference binding, giving: sorted_data = sorted(data, key=?it.sort_key) given ?it=: def sort_key(item): return item.attr1, item.attr2 (A potential bonus of that spelling is that it may be possible to make "given ?it=:" the syntactic keyword introducing the suite, allowing "given" itself to continue to be used as a variable name) Another possible extension would be to combine it references with `as` clauses on if statements and while loops: if (?it=pattern.match(data)) is not None as matched: ... while (?it=pattern.match(data)) is not None as matched: ... == Why not arbitrary embedded assignments? == Primarily because embedded assignments are inherently hard to read, especially in long expressions. Restricting things to one pronoun, and then pursuing PEP 3150's given clause in order to expand to multiple statement local names should help nudge folks towards breaking things up into multiple statements rather than writing ever more complex one-liners. That said, the ?-prefix notation is deliberately designed such that it *could* be used with arbitrary identifiers rather then being limited to a single specific keyword, and the explicit lack of closure support means that there wouldn't be any complex nested scope issues associated with lambda expressions, generator expressions, or container comprehensions. With that approach, "?it" would just be an idiomatic default name like "self" or "cls" rather than being a true keyword. Given arbitrary identifier support, some of the earlier examples might instead be written as: value = ?f() if (?f=calculate) is not None else default range((?start=calculate_start()), ?start+10) value if (?lower=lower_bound()) <= value < ?lower+tolerance else 0 The main practical downside to this approach is that *all* the semantic weight ends up resting on the symbolic "?" prefix, which makes it very difficult to look up as a new Python user. With a keyword embedded in the construct, there's a higher chance that folks will be able to guess the right term to search for (i.e. "python it expression" or "python it keyword"). Another downside of this more flexible option is that it likely *wouldn't* be amenable to the "if expr as name:" syntax extension, as there wouldn't be a single defined pronoun expression to bind the name to. However, the extension to PEP 3150 would allow the statement local namespace to be given an arbitrary name: sorted_data = sorted(data, key=?ns.sort_key) given ?ns=: def sort_key(item): return item.attr1, item.attr2 Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia [View Less]

3 2

Re: [Python-ideas] Coming up with an alternative to PEP 505's None-aware operators
by rymg19＠gmail.com Feb. 19, 2018

Feb. 19, 2018

I don't know...to me this looks downright ugly and an awkward special case. It feels like it combines reading difficulty of inline assignment with the awkwardness of a magic word and the ugliness of using ?. Basically, every con of the other proposals combined... -- Ryan (ライアン) Yoko Shimomura, ryo (supercell/EGOIST), Hiroyuki Sawano >> everyone elsehttps://refi64.com/ On Feb 15, 2018 at 8:07 PM, <Nick Coghlan <ncoghlan(a)gmail.com>> wrote: The recent thread on variable … [View More]assignment in comprehensions has prompted me to finally share https://gist.github.com/ncoghlan/a1b0482fc1ee3c3a11fc7ae64833a315 with a wider audience (see the comments there for some notes on iterations I've already been through on the idea). == The general idea == The general idea would be to introduce a *single* statement local reference using a new keyword with a symbolic prefix: "?it" * `(?it=expr)` is a new atomic expression for an "it reference binding" (whitespace would be permitted around "?it" and "=", but PEP 8 would recommend against it in general) * subsequent subexpressions (in execution order) can reference the bound subexpression using `?it` (an "it reference") * `?it` is reset between statements, including before entering the suite within a compound statement (if you want a persistent binding, use a named variable) * for conditional expressions, put the reference binding in the conditional, as that gets executed first * to avoid ambiguity, especially in function calls (where it could be confused with keyword argument syntax), the parentheses around reference bindings are always required * unlike regular variables, you can't close over statement local references (the nested scope will get an UnboundLocalError if you try it) The core inspiration here is English pronouns (hence the choice of keyword): we don't generally define arbitrary terms in the middle of sentences, but we *do* use pronouns to refer back to concepts introduced earlier in the sentence. And while it's not an especially common practice, pronouns are sometimes even used in a sentence *before* the concept they refer to ;) If we did pursue this, then PEPs 505, 532, and 535 would all be withdrawn or rejected (with the direction being to use an it-reference instead). == Examples == `None`-aware attribute access: value = ?it.strip()[4:].upper() if (?it=var1) is not None else None `None`-aware subscript access: value = ?it[4:].upper() if (?it=var1) is not None else None `None`-coalescense: value = ?it if (?it=var1) is not None else ?it if (?it=var2) is not None else var3 `NaN`-coalescence: value = ?it if not math.isnan((?it=var1)) else ?it if not math.isnan((?that=var2)) else var3 Conditional function call: value = ?it() if (?it=calculate) is not None else default Avoiding repeated evaluation of a comprehension filter condition: filtered_values = [?it for x in keys if (?it=get_value(x)) is not None] Avoiding repeated evaluation for range and slice bounds: range((?it=calculate_start()), ?it+10) data[(?it=calculate_start()):?it+10] Avoiding repeated evaluation in chained comparisons: value if (?it=lower_bound()) <= value < ?it+tolerance else 0 Avoiding repeated evaluation in an f-string: print(f"{?it=get_value()!r} is printed in pure ASCII as {?it!a} and in Unicode as {?it}" == Possible future extensions == One possible future extension would be to pursue PEP 3150, treating the nested namespace as an it reference binding, giving: sorted_data = sorted(data, key=?it.sort_key) given ?it=: def sort_key(item): return item.attr1, item.attr2 (A potential bonus of that spelling is that it may be possible to make "given ?it=:" the syntactic keyword introducing the suite, allowing "given" itself to continue to be used as a variable name) Another possible extension would be to combine it references with `as` clauses on if statements and while loops: if (?it=pattern.match(data)) is not None as matched: ... while (?it=pattern.match(data)) is not None as matched: ... == Why not arbitrary embedded assignments? == Primarily because embedded assignments are inherently hard to read, especially in long expressions. Restricting things to one pronoun, and then pursuing PEP 3150's given clause in order to expand to multiple statement local names should help nudge folks towards breaking things up into multiple statements rather than writing ever more complex one-liners. That said, the ?-prefix notation is deliberately designed such that it *could* be used with arbitrary identifiers rather then being limited to a single specific keyword, and the explicit lack of closure support means that there wouldn't be any complex nested scope issues associated with lambda expressions, generator expressions, or container comprehensions. With that approach, "?it" would just be an idiomatic default name like "self" or "cls" rather than being a true keyword. Given arbitrary identifier support, some of the earlier examples might instead be written as: value = ?f() if (?f=calculate) is not None else default range((?start=calculate_start()), ?start+10) value if (?lower=lower_bound()) <= value < ?lower+tolerance else 0 The main practical downside to this approach is that *all* the semantic weight ends up resting on the symbolic "?" prefix, which makes it very difficult to look up as a new Python user. With a keyword embedded in the construct, there's a higher chance that folks will be able to guess the right term to search for (i.e. "python it expression" or "python it keyword"). Another downside of this more flexible option is that it likely *wouldn't* be amenable to the "if expr as name:" syntax extension, as there wouldn't be a single defined pronoun expression to bind the name to. However, the extension to PEP 3150 would allow the statement local namespace to be given an arbitrary name: sorted_data = sorted(data, key=?ns.sort_key) given ?ns=: def sort_key(item): return item.attr1, item.attr2 Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia _______________________________________________ Python-ideas mailing list Python-ideas(a)python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ [View Less]

4 6

(no subject)
by guido Feb. 19, 2018

Feb. 19, 2018

1 0

(no subject)
by guido Feb. 19, 2018

Feb. 19, 2018

1 0

Re: [Python-ideas] Temporary variables in comprehensions
by Robert Vanden Eynde Feb. 18, 2018

Feb. 18, 2018

Hello, talking about this syntax : [y+2 for x in range(5) let y = x+1] *Previous talks* I've had almost exactly the same idea in June 2017, see subject "variable assignment in functional context here: https://mail.python.org/ pipermail/python-ideas/2017-June/subject.html. Currently this can easily be done by iterating over a list of size 1: [y+2 for x in range(5) for y in [x+1]] (Other ways exist, see section "current possibilities" below). This comparison would answer a lot of questions … [View More]like "can we write [x+2 for x in range(5) for x in [x+1]], so [x+2 for x in range(5) let x = x+1]" the new variable would indeed shadow the old one. In June 2017 I introduced the "let" syntax using the existing "for" keyword, using the "=" instead of a "in" like this: [y+2 for x in range(5) for y = x+1] The only difference I introduced was that it would be logical to accept the syntax in any expression: x = 5 print(y+2 for y = x + 1) or with the "let" keyword: x = 5 print(y+2 let y = x + 1) *Previous talk: Pep* In the June conversation one conclusion was that someone wrote a pep ( https://www.python.org/dev/peps/pep-3150/) in 2010 that's still pending (not exactly the same syntax but the same idea, he used a "given:" syntax) that looked like that : print(y+2 given: y=x+1) *Previous talk: GitHub implementation on Cython* Another conclusion was that someone on GitHub implemented a "where" statement in Cython (url: https://github.com/thektulu/cpython/commit/ 9e669d63d292a639eb6ba2ecea3ed2c0c23f2636) where one could write : print(y+2 where y = x+1) So in a list comprehension: [y+2 where y = x + 1 for x in range(5)] As the author thektulu said "just compile and have fun". *New syntax in list comprehension* However, we probably would like to be able to use this "where" after the for: [y+2 for x in range(5) where y = x+1] This would allow the new variable to be used in further "for" and "if" statement : [y+z for x in range(5) where y = x+1 for z in range(y+1)] *Choice of syntax* Initially I thought re-using the "for" keyword would be a good idea for backward comptability (a variable named "where" in old code wouldn't be a problem), however some people pointed out that the python Grammar wouldn't be LL1, when reading the "for" token it wouldn't be able to choose directly if the rest would be a "for in" or "for =" so actually introducing a dedicated keyword is probably better, like this : [y+2 for x in range(5) where y = x+1] print(y+2 where y = x+1) The "where" keyword is very readable in my opinion, very close to English sentences we use. Sometimes we introduce a new variable before using it, generally using "let" or after using it using "where". For example "let y = x+1; print(y+2)". Or "print(y+2 where y = x+1)". The first syntax is chosen in the "let in" syntax in haskell : print(let y = x+2 in y+2) Or chained: print(let x = 2 in let y = x+1 in y+2) But Haskell user would probably line break for clarity : print(let x = 2 in let y = x+1 in y+2) A postfix notation using "where" would probably be less verbose in my opinion. Another example of "postfix notation" python has is with the "a if condition else b" so adding a new one wouldn't be surprising. Furthermore, the postfix notation is preferred in the context of "presenting the result first, then the implementation" (context discussed already in the 2010 pep), the "presenting the result first" is also a goal of the list comprehension, indeed one does write [x+3 for x in range(5)] and not [for x in range(5): x+3], the latter would be more "imperative programming" style, and would be translated to a normal loop. The problem of chaining without parenthesis, how to remove the parenthesis in the following statement ? print((y+2 where y = x+1) where x = 2) We have two options : print(y+2 where x = 2 where y = x+1) print(y+2 where y = x+1 where x = 2) The first option would be probably closer to the way multiple "for" are linked in a list comprehension: [y+2 for x in range(5) for y in [x+1]] But the second option would be more "present result first" and more close to the parenthesized version, the user would create new variables as they go, "I want to compute y+2 but hey, what is y ? It's x+1 ! But what is x ? It's 5 !)). However, keeping the same order as in "multiple for in list comprehension" is better, so I'd choose first option. In the implementation on GitHub of thektulu the parenthesis are mandatory. Another syntax issue would probably surprise some users, the following statement, parenthesized : [(y+2 where y = x+1) for x in range(5)] Would have two totally legal ways to be done: [y+2 where y = x+1 for x in range(5)] [y+2 for x in range(5) where y = x+1] The first one is a consequence of the "where" keyword usable in an expression and the second one is a consequence of using it in a list comprehension. However I think it doesn't break the "there is only one obvious way to do it", because depending on the case, the "y" variable would be a consequence of the iteration or a consequence of the computation. *Goals of the new syntax* Personally, I find it very useful to do an assignment in such context, I use list comprehension for example to generate a big json with multiple for and if. I don't have here a list of big real world example thar would be simplified using this syntax but people interested could search arguments here. People could argue only the "list comprehension" case would be useful and not the "any expression" case: [y+2 for x in range(5) where y = x+1] would be accepted but not : print(y+2 where y=x+1) Because the latter could be written: y = x + 1 print(y+2) However, the idea of having an isolated scope can be a good idea. *Current possibilities* Currently we have multiple options : [y+2 for x in range(5) for y in [x+1]] [y+2 for (x+1 for x in range(5))] [(lambda y:y+2)(y=x+1) for x in range(5)] The first one works well but it's not obvious we just want to assign a new variable, especially when the expression is long, or multiple, or both: [y+z for x in range(5) for y,z in [((x + 1) * 2, x ** 2 - 5)]] The second one makes it impossible to reuse the "x" variable and the y = x+1 relation is not obvious. The third example is what a functional programmer would think but is really too much complex for a beginner and very verbose. *Proposed syntax : Conclusion* As I said, I like the "where" syntax with the "where" keyword. [y+2 for x in range(5) where y = x+1] Also usable in any expression : print(y+2 where y = x+1) *Conclusion* Here is all the talk/work/argument I've already found about this syntax. Apparently it's been a while (2010) since such an idea was thought but I think having a new pep listing all the pros and cons would be a good idea. So that we can measure how much the community would want this concept to be introduced, and if it's refused, the community would have a document where the "cons" are clearly written. Robert Vanden Eynde [View Less]

2 1

Give ipaddresses an __index__ method
by Steven D'Aprano Feb. 15, 2018

Feb. 15, 2018

This idea is inspired by Eric Osborne's post "Extending __format__ method in ipaddress", but I wanted to avoid derailing that thread. I notice what seems to be an inconsistency in the ipaddress objects: py> v4 = ipaddress.IPv4Address('1.2.3.4') py> bin(v4) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: 'IPv4Address' object cannot be interpreted as an integer But that's surely not right: we just need to explicitly do so: py> bin(int(… [View More]

4 6

Extending __format__ method in ipaddress
by Eric Osborne Feb. 15, 2018

Feb. 15, 2018

Folks- The ipaddress library returns an IP address object which can represent itself in a number of ways: In [1]: import ipaddress In [2]: v4 = ipaddress.IPv4Address('1.2.3.4') In [3]: print(v4) 1.2.3.4 In [4]: v4 Out[4]: IPv4Address('1.2.3.4') In [6]: v4.packed Out[6]: b'\x01\x02\x03\x04' In [9]: str(v4) Out[9]: '1.2.3.4' In [10]: int(v4) Out[10]: 16909060 In [13]: bin(int(v4)) Out[13]: '0b1000000100000001100000100' In [14]: hex(int(v4)) Out[14]: '0x1020304' In [15]: oct(int(v4)) … [View More]Out[15]: '0o100401404' There are IPv6 objects as well: In [6]: v6 = ipaddress.IPv6Address('2001:0db8:85a3:0000:0000:8a2e:0370:7334') In [7]: int(v6) Out[7]: 42540766452641154071740215577757643572 and what I'm proposing will work for both address families. In either case, bin/hex/oct don't work on them directly, but on the integer representation. This is a little annoying but not such a big deal. What is a big deal (at least to me) is that the binary representation isn't zero-padded. This makes it harder to compare two IP addresses by eye to see what the differences are, i.e.: In [16]: a = ipaddress.IPv4Address('0.2.3.4') In [30]: bin(int(a)) Out[30]: '0b100000001100000100' In [31]: bin(int(v4)) Out[31]: '0b1000000100000001100000100' It would be nice if there was a way to have an IP address always present itself in fully zero-padded binary (32 bits for IPv4, 128 bits for IPv6). I find this particularly convenient when putting together training material, as it's easier to show subnetting and aggregation if you point at the binary than if you give people dotted-quad addresses and ask them to do the binary conversion in their head. Hex is also handy when you're comparing a dotted-quad IP address to a hex sniffer trace. It's possible to do this in a one-liner (thanks to Eric Smith): f'{int(v4):#0{34}b}'. But this is a little cryptic. I opened bpo-32820 (https://github.com/python/cpython/pull/5627) to contribute a way to do this. I started with an __index__ method but Issue 15559 ( https://github.com/python/cpython/commit/e0c3f5edc0f20cc28363258df501758c1b…) rules this out. I instead added a bits() class method so that v4.bits would return the fully padded string. This was not terribly pretty, but it mirrored packed(), at least. Nick Coghlan suggested I instead extend __format__, which is what the diffs in the current pull request do. This allows a great deal more flexibility: the current code takes 'b', 'n', or 'x' types, as well as the '#' option and support for the '_' separator. I realize now I didn't add 'o' but I certainly can for completeness. I debated adding rfc1924 encoding for IPv6 addresses but decided it was entirely too silly. This is just a convenience function, but IMO fills a need. Is this worth pursuing? eric [View Less]

3 2

Complicate str methods
by Franklin? Lee Feb. 13, 2018

Feb. 13, 2018

Let s be a str. I propose to allow these existing str methods to take params in new forms. s.replace(old, new): Allow passing in a collection of olds. Allow passing in a single argument, a mapping of olds to news. Allow the olds in the mapping to be tuples of strings. s.split(sep), s.rsplit, s.partition: Allow sep to be a collection of separators. s.startswith, s.endswith: Allow argument to be a collection of strings. s.find, s.index, s.count, x in s: Similar. … [View More]

11 14