Mailman 3 August 2014 - Python-ideas

@classproperty, @abc.abstractclasspropery, etc.
by K. Richard Pixley 16 Dec '20

16 Dec '20

There's a whole matrix of these and I'm wondering why the matrix is currently sparse rather than implementing them all. Or rather, why we can't stack them as: class foo(object): @classmethod @property def bar(cls, ...): ... Essentially the permutation are, I think: {'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable attribute}. concreteness implicit first arg type name comments {unadorned} {unadorned} method def foo(): exists now {unadorned} {unadorned} property @property exists now {unadorned} {unadorned} non-callable attribute x = 2 exists now {unadorned} static method @staticmethod exists now {unadorned} static property @staticproperty proposing {unadorned} static non-callable attribute {degenerate case - variables don't have arguments} unnecessary {unadorned} class method @classmethod exists now {unadorned} class property @classproperty or @classmethod;@property proposing {unadorned} class non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract {unadorned} method @abc.abstractmethod exists now abc.abstract {unadorned} property @abc.abstractproperty exists now abc.abstract {unadorned} non-callable attribute @abc.abstractattribute or @abc.abstract;@attribute proposing abc.abstract static method @abc.abstractstaticmethod exists now abc.abstract static property @abc.staticproperty proposing abc.abstract static non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract class method @abc.abstractclassmethod exists now abc.abstract class property @abc.abstractclassproperty proposing abc.abstract class non-callable attribute {degenerate case - variables don't have arguments} unnecessary I think the meanings of the new ones are pretty straightforward, but in case they are not... @staticproperty - like @property only without an implicit first argument. Allows the property to be called directly from the class without requiring a throw-away instance. @classproperty - like @property, only the implicit first argument to the method is the class. Allows the property to be called directly from the class without requiring a throw-away instance. @abc.abstractattribute - a simple, non-callable variable that must be overridden in subclasses @abc.abstractstaticproperty - like @abc.abstractproperty only for @staticproperty @abc.abstractclassproperty - like @abc.abstractproperty only for @classproperty --rich

10 15

Specify number of items to allocate for array.array() constructor
by Sven Rahmann 22 Feb '20

22 Feb '20

At the moment, the array module of the standard library allows to create arrays of different numeric types and to initialize them from an iterable (eg, another array). What's missing is the possiblity to specify the final size of the array (number of items), especially for large arrays. I'm thinking of suffix arrays (a text indexing data structure) for large texts, eg the human genome and its reverse complement (about 6 billion characters from the alphabet ACGT). The suffix array is a long int array of the same size (8 bytes per number, so it occupies about 48 GB memory). At the moment I am extending an array in chunks of several million items at a time at a time, which is slow and not elegant. The function below also initializes each item in the array to a given value (0 by default). Is there a reason why there the array.array constructor does not allow to simply specify the number of items that should be allocated? (I do not really care about the contents.) Would this be a worthwhile addition to / modification of the array module? My suggestions is to modify array generation in such a way that you could pass an iterator (as now) as second argument, but if you pass a single integer value, it should be treated as the number of items to allocate. Here is my current workaround (which is slow): def filled_array(typecode, n, value=0, bsize=(1<<22)): """returns a new array with given typecode (eg, "l" for long int, as in the array module) with n entries, initialized to the given value (default 0) """ a = array.array(typecode, [value]*bsize) x = array.array(typecode) r = n while r >= bsize: x.extend(a) r -= bsize x.extend([value]*r) return x

14 20

Implicit string literal concatenation considered harmful?
by Guido van Rossum 14 Mar '18

14 Mar '18

I just spent a few minutes staring at a bug caused by a missing comma -- I got a mysterious argument count error because instead of foo('a', 'b') I had written foo('a' 'b'). This is a fairly common mistake, and IIRC at Google we even had a lint rule against this (there was also a Python dialect used for some specific purpose where this was explicitly forbidden). Now, with modern compiler technology, we can (and in fact do) evaluate compile-time string literal concatenation with the '+' operator, so there's really no reason to support 'a' 'b' any more. (The reason was always rather flimsy; I copied it from C but the reason why it's needed there doesn't really apply to Python, as it is mostly useful inside macros.) Would it be reasonable to start deprecating this and eventually remove it from the language? -- --Guido van Rossum (python.org/~guido)

51 165

Negative limit for traceback.*
by Thomas Allen 12 Oct '14

12 Oct '14

I'd like to propose adding the ability to extract last n stack trace entries from a traceback using functions from traceback module. Simply put, passing limit=-n would make functions to return (or print or yield) last abs(n) entries. No-brainer implementation for extract_tb: extracted = list(_extract_tb_iter(tb, limit=(limit, None)[limit < 0])) if limit is not None and limit < 0: return extracted[limit:] return extracted The motivation: http://stackoverflow.com/q/25472412/2301450

3 3

Better integration of multiprocessing with asyncio
by Dan O'Reilly 09 Oct '14

09 Oct '14

I think it would be helpful for folks using the asyncio module to be able to make non-blocking calls to objects in the multiprocessing module more easily. While some use-cases for using multiprocessing can be replaced with ProcessPoolExecutor/run_in_executor, there are others that cannot; more advanced usages of multiprocessing.Pool aren't supported by ProcessPoolExecutor (initializer/initargs, contexts, etc.), and other multiprocessing classes like Lock and Queue have blocking methods that could be made into coroutines. Consider this (extremely contrived, but use your imagination) example of a asyncio-friendly Queue: import asyncio import time def do_proc_work(q, val, val2): time.sleep(3) # Imagine this is some expensive CPU work. ok = val + val2 print("Passing {} to parent".format(ok)) q.put(ok) # The Queue can be used with the normal blocking API, too. item = q.get() print("got {} back from parent".format(item)) def do_some_async_io_task(): # Imagine there's some kind of asynchronous I/O # going on here that utilizes asyncio. asyncio.sleep(5) @asyncio.coroutine def do_work(q): loop.run_in_executor(ProcessPoolExecutor(), do_proc_work, q, 1, 2) do_some_async_io_task() item = yield from q.coro_get() # Non-blocking get that won't affect our io_task print("Got {} from worker".format(item)) item = item + 25 yield from q.coro_put(item) if __name__ == "__main__": q = AsyncProcessQueue() # This is our new asyncio-friendly version of multiprocessing.Queue loop = asyncio.get_event_loop() loop.run_until_complete(do_work(q)) I have seen some rumblings about a desire to do this kind of integration on the bug tracker (http://bugs.python.org/issue10037#msg162497 and http://bugs.python.org/issue9248#msg221963) though that discussion is specifically tied to merging the enhancements from the Billiard library into multiprocessing.Pool. Are there still plans to do that? If so, should asyncio integration with multiprocessing be rolled into those plans, or does it make sense to pursue it separately? Even more generally, do people think this kind of integration is a good idea to begin with? I know using asyncio is primarily about *avoiding* the headaches of concurrent threads/processes, but there are always going to be cases where CPU-intensive work is going to be required in a primarily I/O-bound application. The easier it is to for developers to handle those use-cases, the better, IMO. Note that the same sort of integration could be done with the threading module, though I think there's a fairly limited use-case for that; most times you'd want to use threads over processes, you could probably just use non-blocking I/O instead. Thanks, Dan

4 10

keyword for introducing generators
by Neil Girdhar 20 Sep '14

20 Sep '14

I'm sure this has been suggested before, but I just spent two days trying to figure out why a method wasn't being called only to find that I'd accidentally pasted a yield into the function. What is the argument against a different keyword for introducing generator functions/methods? If it's backward compatibility, then my suggestion to have a from __future__ and then make it real in Python 4. Neil

5 6

Subprocess: Add an encoding argument
by Paul Moore 03 Sep '14

03 Sep '14

At the moment, subprocess offers two options for handlingthe standard IO streams of the child. By default, the streams are binary, or you can set universal_newlines to get text-mode streams with universal newline handling enabled. With universal_newlines, the encoding of the streams is the default value for the environment (whatever locale.getpreferredencoding() returns). However, there can be cases where you want finer control over the encoding to use (for example, if you run Python in a subprocess and set PYTHONIOENCODING). I propose adding an "encoding" parameter to subprocess.Popen (and the various wrapper routines) to allow specifying the actual encoding to use. Obviously, you can simply wrap the binary streams yourself - the main use for this facility would be in the higher level functions like check_output and communicate. Does this seem like a reasonable suggestion? Paul

8 20

List available tests without running them
by Thomas Allen 31 Aug '14

31 Aug '14

I'd like to propose the ability to list all available unit tests without executing them by providing a specific command line option (-l/--list for instance). python3 -m unittest discover -ls . would list all tests in current directory. Motivation & possible implementation: http://stackoverflow.com/q/24478727/2301450

1 0

Indentation, again: Problems with moving and combining code across editors.
by Milind Khadilkar 30 Aug '14

30 Aug '14

Hi! I do appreciate that indentation in Python code shows intention, and I am all for it, but problems do arise if code is copied from one place to another, even from specialized Python editors.... and there is little you can do to recover intentions/indentations lost in the process. Besides, spaces become difficult to count and distinguish from tabs, adding to the problem. (Why allow tabs at all?) Why can't Python be made to accept the following indentation of code (in *addition* to the current schemes)? There could be some flexibility of choosing the indentation character. (Pipe was mentioned as an alternative in another thread) for i in range(10): .if i>5: ..print 10-i .else: ..print i print "!" If not in the interpreter for some reason, could it be "advised" as a mode in a python oriented editor, or a "dictated" mechanism for copy/paste/transfer of code? Sorry for posing an elementary question. Thanks and regards MK-zedobject

9 9

Query available codecs and error handlers
by Steven D'Aprano 29 Aug '14

29 Aug '14

Unless I am badly misinformed, there is no way to programmatically check what codecs and error handlers are available. I propose two functions in the codecs module: get_codecs() get_error_handlers() which each return a set of the available codec, or error handler, names. Use-cases --------- (1) At the interactive interpreter, it is often useful to experiment with different codecs. Being able to get a list of available codecs avoids the frustration of having to look them up elsewhere, or guess. (2) Applications which offer to read or write text files may wish to allow the user to select an encoding. With no obvious way of finding out what encodings are available, the application is limited to hard-coding some, and hoping that they don't get renamed or removed. Why sets, rather than lists? ---------------------------- Sets emphasis that the names are in no particular order. Since the names should always be strings and therefore hashable, and the order is irrelevant, there is no particular need for a list. Should they be frozensets rather than sets? ------------------------------------------- I don't care :-) Using sets may make it easier for the application to filter the results in some way, but I really don't think it matters. Thoughts or comments? -- Steven

4 8