Python-ideas February 2017

python-ideas@python.org

79 participants
43 discussions

Specify number of items to allocate for array.array() constructor

by Sven Rahmann

At the moment, the array module of the standard library allows to create arrays of different numeric types and to initialize them from an iterable (eg, another array). What's missing is the possiblity to specify the final size of the array (number of items), especially for large arrays. I'm thinking of suffix arrays (a text indexing data structure) for large texts, eg the human genome and its reverse complement (about 6 billion characters from the alphabet ACGT). The suffix array is a long int array of the same size (8 bytes per number, so it occupies about 48 GB memory). At the moment I am extending an array in chunks of several million items at a time at a time, which is slow and not elegant. The function below also initializes each item in the array to a given value (0 by default). Is there a reason why there the array.array constructor does not allow to simply specify the number of items that should be allocated? (I do not really care about the contents.) Would this be a worthwhile addition to / modification of the array module? My suggestions is to modify array generation in such a way that you could pass an iterator (as now) as second argument, but if you pass a single integer value, it should be treated as the number of items to allocate. Here is my current workaround (which is slow): def filled_array(typecode, n, value=0, bsize=(1<<22)): """returns a new array with given typecode (eg, "l" for long int, as in the array module) with n entries, initialized to the given value (default 0) """ a = array.array(typecode, [value]*bsize) x = array.array(typecode) r = n while r >= bsize: x.extend(a) r -= bsize x.extend([value]*r) return x

7 months

Positional-only parameters

by Victor Stinner

Hi, For technical reasons, many functions of the Python standard libraries implemented in C have positional-only parameters. Example: ------- $ ./python Python 3.7.0a0 (default, Feb 25 2017, 04:30:32) >>> help(str.replace) replace(self, old, new, count=-1, /) # <== notice "/" at the end ... >>> "a".replace("x", "y") # ok 'a' >>> "a".replace(old="x", new="y") # ERR! TypeError: replace() takes at least 2 arguments (0 given) ------- When converting the methods of the builtin str type to the internal "Argument Clinic" tool (tool to generate the function signature, function docstring and the code to parse arguments in C), I asked if we should add support for keyword arguments in str.replace(). The answer was quick: no! It's a deliberate design choice. Quote of Yury Selivanov's message: """ I think Guido explicitly stated that he doesn't like the idea to always allow keyword arguments for all methods. I.e. `str.find('aaa')` just reads better than `str.find(needle='aaa')`. Essentially, the idea is that for most of the builtins that accept one or two arguments, positional-only parameters are better. """ http://bugs.python.org/issue29286#msg285578 I just noticed a module on PyPI to implement this behaviour on Python functions: https://pypi.python.org/pypi/positional My question is: would it make sense to implement this feature in Python directly? If yes, what should be the syntax? Use "/" marker? Use the @positional() decorator? Do you see concrete cases where it's a deliberate choice to deny passing arguments as keywords? Don't you like writing int(x="123") instead of int("123")? :-) (I know that Serhiy Storshake hates the name of the "x" parameter of the int constructor ;-)) By the way, I read that "/" marker is unknown by almost all Python developers, and [...] syntax should be preferred, but inspect.signature() doesn't support this syntax. Maybe we should fix signature() and use [...] format instead? Replace "replace(self, old, new, count=-1, /)" with "replace(self, old, new[, count=-1])" (or maybe even not document the default value?). Python 3.5 help (docstring) uses "S.replace(old, new[, count])". Victor

2 years

Implicit string literal concatenation considered harmful?

by Guido van Rossum

I just spent a few minutes staring at a bug caused by a missing comma -- I got a mysterious argument count error because instead of foo('a', 'b') I had written foo('a' 'b'). This is a fairly common mistake, and IIRC at Google we even had a lint rule against this (there was also a Python dialect used for some specific purpose where this was explicitly forbidden). Now, with modern compiler technology, we can (and in fact do) evaluate compile-time string literal concatenation with the '+' operator, so there's really no reason to support 'a' 'b' any more. (The reason was always rather flimsy; I copied it from C but the reason why it's needed there doesn't really apply to Python, as it is mostly useful inside macros.) Would it be reasonable to start deprecating this and eventually remove it from the language? -- --Guido van Rossum (python.org/~guido)

2 years, 6 months

Re: [Python-ideas] unify usage of mutable and immutable objects

by 语言破碎处

> To me, 'pop' implies mutation. Tuples do not have a pop method, and it > is not obvious to me that either tuples or frozensets should. What are > the use cases that are not better served by converting to list or set? > -- > Terry Jan Reedy 1) coverting to set or list is O(n) in time 2) if I have to keep the old copy, standard set solution will be O(n) both in time and space! working examples: 1) priority queue: insert and pop occur 2) share immutable data to difference subroutines: each one can modify local copy safely and concurrency.

3 years, 5 months

get() method for list and tuples

by Michel Desmoulin

dict.get() is a very useful method, but for lists and tuples, we need to rely on try/except instead. Can we get list.get and tuple.get as well? Also, for list, a list.setdefault like the dict.setdefault would be logical.

3 years, 6 months

add __contains__ into the "type" object

by 语言破碎处

where we use types? almost: isinstance(obj, T); # issubclass(S, T); Note that TYPE is SET; if we add __contains__ and __le__ into "type", then things become: obj in T; # S <= T; # if only not misleading to a total ordering example: def __getitems__(self, i): if i in Integral: ... elif i in slice: ... # Save "(,)". Really, I prefer to type "lambda:;" than "()". # I fail to modify the "type" object, since it is a C-object.

3 years, 6 months

Delayed Execution via Keyword

by Joseph Hackman

Howdy All! This suggestion is inspired by the question on "Efficient debug logging". I propose a keyword to mark an expression for delayed/lazy execution, for the purposes of standardizing such behavior across the language. The proposed format is: delayed: <expr> i.e. log.info("info is %s", delayed: expensiveFunction()) Unlike 'lambda' which returns a function (so the receiver must be lambda-aware), delayed execution blocks are for all purposes values. The first time the value (rather than location) is read, or any method on the delayed object is called, the expression is executed and the delayed expression is replaced with the result. (Thus, the delayed expression is only every evaluated once). Ideally: a = delayed: 1+2 b = a print(a) #adds 1 and 2, prints 3 # a and b are now both just 3 print(b) #just prints 3 Mechanically, this would be similar to the following: class Delayed(): def __init__(self, func): self.__func = func self.__executed = False self.__value = None def __str__(self): if self.__executed: return self.__value.__str__() self.__value = self.__func() self.__executed = True return self.__value.__str__() def function_print(value): print('function_print') print(value) def function_return_stuff(value): print('function_return_stuff') return value function_print(function_return_stuff('no_delay')) function_print(Delayed(lambda: function_return_stuff('delayed'))) delayed = Delayed(lambda: function_return_stuff('delayed_object')) function_print(delayed) function_print(delayed) Unfortunately, due to https://docs.python.org/3/reference/datamodel.html#special-lookup , this magic delayed class would need to implement many magic methods, as __getattribute__ is not _always_ called.

3 years, 6 months

lazy use for optional import

by Nicolas Cellier

I have seen some interest into lazy functionality implementation. I wondered if it can be linked with optional import. PEP 8 <http://www.python.org/dev/peps/pep-0008/> authoritatively states: Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants. So, if we want to stick to PEP8 with non mandatory import, we have to catch the import errors, or jail the class or function using extra functionnality. Why not using the potential lazy keyword to have a nice way to deal with it? For example: lazy import pylab as pl # do nothing for now > > # do stuff > > def plot(*args): > pl.figure() # Will raise an ImportError at this point > pl.plot(...) > That way, our library will raise an ImportError only on plot func usage with an explicit traceback : if matplotlib is not installed, we will have the line where it is used for the first time and we will have the name of the faulty library.

3 years, 6 months

suggestion about the sort() function of the list instance

by qhlonline

Hi, all I have a suggestion that, the sort() member method of the list instance, should return the 'self' as the result of list.sort() call. Now list.sort() returns nothing, so that I can NOT write code like this: res = {item: func(item) for item in item_list.sort()} It feels bad. Regards!

3 years, 6 months

__repr__: to support pprint

by 语言破碎处

Hi, everyone! Oftenly, __repr__ return "{type}({args}, {kws})" problem: 1) to call .format() myself is tedious I have seen someone do it wrong: "T(1,)"! I write a helper function myself, but to import such tiny function everywhere isnot good. 2) pprint cannot dive into string that from repr() To use pprint, I sometimes have to recursively turn objects into builtin containers: (type_name, args...) solution: allow __repr__ to return str OR tuple: (args, kws) Better, isn't it?

3 years, 6 months

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas February 2017