Python-ideas June 2014

python-ideas@python.org

57 participants
24 discussions

Specify number of items to allocate for array.array() constructor

by Sven Rahmann

At the moment, the array module of the standard library allows to create arrays of different numeric types and to initialize them from an iterable (eg, another array). What's missing is the possiblity to specify the final size of the array (number of items), especially for large arrays. I'm thinking of suffix arrays (a text indexing data structure) for large texts, eg the human genome and its reverse complement (about 6 billion characters from the alphabet ACGT). The suffix array is a long int array of the same size (8 bytes per number, so it occupies about 48 GB memory). At the moment I am extending an array in chunks of several million items at a time at a time, which is slow and not elegant. The function below also initializes each item in the array to a given value (0 by default). Is there a reason why there the array.array constructor does not allow to simply specify the number of items that should be allocated? (I do not really care about the contents.) Would this be a worthwhile addition to / modification of the array module? My suggestions is to modify array generation in such a way that you could pass an iterator (as now) as second argument, but if you pass a single integer value, it should be treated as the number of items to allocate. Here is my current workaround (which is slow): def filled_array(typecode, n, value=0, bsize=(1<<22)): """returns a new array with given typecode (eg, "l" for long int, as in the array module) with n entries, initialized to the given value (default 0) """ a = array.array(typecode, [value]*bsize) x = array.array(typecode) r = n while r >= bsize: x.extend(a) r -= bsize x.extend([value]*r) return x

8 months, 1 week

Implicit string literal concatenation considered harmful?

by Guido van Rossum

I just spent a few minutes staring at a bug caused by a missing comma -- I got a mysterious argument count error because instead of foo('a', 'b') I had written foo('a' 'b'). This is a fairly common mistake, and IIRC at Google we even had a lint rule against this (there was also a Python dialect used for some specific purpose where this was explicitly forbidden). Now, with modern compiler technology, we can (and in fact do) evaluate compile-time string literal concatenation with the '+' operator, so there's really no reason to support 'a' 'b' any more. (The reason was always rather flimsy; I copied it from C but the reason why it's needed there doesn't really apply to Python, as it is mostly useful inside macros.) Would it be reasonable to start deprecating this and eventually remove it from the language? -- --Guido van Rossum (python.org/~guido)

2 years, 7 months

Re: [Python-ideas] Empty set, Empty dict

by Wichert Akkerman

Victor Stinner wrote: 2014-06-10 8:15 GMT+02:00 Neil Girdhar <mistersheik at gmail.com>: > > I've seen this proposed before, and I personally would love this, but my > > guess is that it breaks too much code for too little gain. > > > > On Wednesday, May 21, 2014 12:33:30 PM UTC-4, Frédéric Legembre wrote: > >> > >> > >> Now | Future | > >> ---------------------------------------------------- > >> () | () | empty tuple ( 1, 2, 3 ) > >> [] | [] | empty list [ 1, 2, 3 ] > >> set() | {} | empty set { 1, 2, 3 } > >> {} | {:} | empty dict { 1:a, 2:b, 3:c } > > > Your guess is right. It will break all Python 2 and Python 3 in the world. > > Technically, set((1, 2)) is different than {1, 2}: the first creates a > tuple and loads the global name "set" (which can be replaced at > runtime!), whereas the later uses bytecode and only store values > (numbers 1 and 2). > > It would be nice to have a syntax for empty set, but {} is a no-no. Perhaps {,} would be a possible spelling. For consistency you might want to allow (,) to create an empty tuple as well; personally I would find that more intuitive that (()). Wichert.

6 years, 4 months

Special keyword denoting an infinite loop

by Thomas Allen

Rust language defines a special way to make an infinite loop ( http://doc.rust-lang.org/tutorial.html#loops). I propose adding the same keyword to Python. It will be very useful for WSGI servers and will suit as a more convenient replacement for recursion (hence Python doesn't do TRE). I personally find it much prettier than *while True* or *while 1*. It won't cause any problems with existing programs, because *loop* is very rarely used as a variable name. For instance while True: > do_something() > do_something_else() would turn to loop: > do_something() > do_something_else()

6 years, 4 months

Re: [Python-ideas] Special keyword denoting an infinite loop

by Nick Coghlan

On 29 June 2014 00:56, Benny Khoo <benny_khoo_99(a)yahoo.com> wrote: > rather than a special keyword in Python, how about having Python to support > the concept of passing block (a group of statements) as argument? I thought > that can be quite elegant solution. So a loop statement can be interpreted > simply as a function that accept a block e.g. loop [block]? > > Supporting block has a lot of practical applications. I remember seeing some > special purpose flow control functions as early as Tcl. We also see it in > Ruby and the more recently the new Swift language. This is a well worn path, and it's difficult to retrofit to an existing language. Ruby, at least, relies heavily on a convention of taking blocks as the last argument to a function to make things work, which is a poor fit to Python's keyword arguments and far more varied positional signatures for higher order functions. PEP 403 and PEP 3150 are a couple of different explorations of the idea a more block-like feature. http://python-notes.curiousefficiency.org/en/latest/pep_ideas/suite_expr.ht… is one that goes even further to consider a delineated subsyntax for Python that would allow entire suites as expressions. However, the stumbling block all these proposals tend to hit is that proponents really, really, struggle to come up with compelling use cases where "just define a named function" isn't a clearer and easier to understand answer. Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

6 years, 4 months

problems with import

by Neal Becker

One problem I often encounter is with import search. An example of the problem is with the package mercurial. It has extensions in a subdirectory called 'hgext'. On fedora, I install mercurial using the vendor package. This creates /usr/lib64/python2.7/site-packages/hgext/ Later, I want to try out an extension as a non-privileged user. cd <extension> python setup.py install --user Now I also have ~/.local/lib/python2.7/site-packages/hgext but python won't search there for extensions. Once if finds the system hgext directory, it won't look also in the local one. Any thoughts?

6 years, 4 months

Accepting keyword arguments for __getitem__

by Stefano Borini

Dear all, At work we use a notation like LDA[Z=5] to define a specific level of accuracy for our evaluation. This notation is used just for textual labels, but it would be nice if it actually worked at the scripting level, which led me to think to the following: at the moment, we have the following >>> class A: ... def __getitem__(self, y): ... print(y) ... >>> a=A() >>> a[2] 2 >>> a[2,3] (2, 3) >>> a[1:3] slice(1, 3, None) >>> a[1:3, 4] (slice(1, 3, None), 4) >>> I would propose to add the possibility for a[Z=3], where y would then be a dictionary {"Z": 3}. In the case of a[1:3, 4, Z=3, R=5], the value of y would be a tuple containing (slice(1,3,None), 4, {"Z": 3}, {"R": 5}). This allows to preserve the ordering as specified (e.g. a[Z=3, R=4] vs a[R=4, Z=3]). Do you think it would be a good/useful idea? Was this already discussed or proposed in a PEP? Google did not help on this regard. Thank you, Stefano Borini

6 years, 4 months

as_completed

by Ram Rachum

What do you think about an argument as_completed=False to Executor.map ? Personally I'd find it really handy.

6 years, 4 months

Another pathlib suggestion

by Antony Lee

Handling of Paths with multiple extensions is currently not so easy with pathlib. Specifically, I don't think there is an easy way to go from "foo.tar.gz" to "foo.ext", because Path.with_suffix only replaces the last suffix. I would therefore like to suggest either 1/ add Path.replace_suffix, such that Path("foo.tar.gz").replace_suffix(".tar.gz", ".ext") == Path("foo.ext") (this would also provide extension-checking capabilities, raising ValueError if the first argument is not a valid suffix of the initial path); or 2/ add a second argument to Path.with_suffix, "n_to_strip" (although perhaps with a better name), defaulting to 1, such that Path("foo.tar.gz").with_suffix(".ext", 0) == Path("foo.tar.gz.ext") Path("foo.tar.gz").with_suffix(".ext", 1) == Path("foo.tar.ext") Path("foo.tar.gz").with_suffix(".ext", 2) == Path("foo.ext") # set n_to_strip to len(path.suffixes) for stripping all of them. Path("foo.tar.gz").with_suffix(".ext", 3) raises a ValueError. Best, Antony

6 years, 4 months

"Escape hatch" for preferred encoding (default encoding for `open`)

by Ionel Maries Cristian

It would be nice if there would be an escape hatch for situations where the value of locale.getpreferredencoding() can't be changed (eg: windows - try changing that to utf8 <http://blog.ionelmc.ro/2014/06/19/just-another-day-using-python-3/>) in the form of an environment variable like PYTHONPREFERREDENCODING (or something like that). The idea is that it would override the default encoding for open() for platforms/situations where it's infeasible to manually specify the encoding to open (eg: lots of old code) or change locale to something utf8-ish (windows). I've found an old thread <http://grokbase.com/t/python/python-dev/116w24gdra/open-set-the-default-enc…> about this problem but to my bewilderment no one considered using an environment variable. Thanks, -- Ionel M.

6 years, 4 months

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

Python-ideas June 2014