Mailman 3 July 2011 - Python-ideas

@classproperty, @abc.abstractclasspropery, etc.
by K. Richard Pixley 16 Dec '20

16 Dec '20

There's a whole matrix of these and I'm wondering why the matrix is currently sparse rather than implementing them all. Or rather, why we can't stack them as: class foo(object): @classmethod @property def bar(cls, ...): ... Essentially the permutation are, I think: {'unadorned'|abc.abstract}{'normal'|static|class}{method|property|non-callable attribute}. concreteness implicit first arg type name comments {unadorned} {unadorned} method def foo(): exists now {unadorned} {unadorned} property @property exists now {unadorned} {unadorned} non-callable attribute x = 2 exists now {unadorned} static method @staticmethod exists now {unadorned} static property @staticproperty proposing {unadorned} static non-callable attribute {degenerate case - variables don't have arguments} unnecessary {unadorned} class method @classmethod exists now {unadorned} class property @classproperty or @classmethod;@property proposing {unadorned} class non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract {unadorned} method @abc.abstractmethod exists now abc.abstract {unadorned} property @abc.abstractproperty exists now abc.abstract {unadorned} non-callable attribute @abc.abstractattribute or @abc.abstract;@attribute proposing abc.abstract static method @abc.abstractstaticmethod exists now abc.abstract static property @abc.staticproperty proposing abc.abstract static non-callable attribute {degenerate case - variables don't have arguments} unnecessary abc.abstract class method @abc.abstractclassmethod exists now abc.abstract class property @abc.abstractclassproperty proposing abc.abstract class non-callable attribute {degenerate case - variables don't have arguments} unnecessary I think the meanings of the new ones are pretty straightforward, but in case they are not... @staticproperty - like @property only without an implicit first argument. Allows the property to be called directly from the class without requiring a throw-away instance. @classproperty - like @property, only the implicit first argument to the method is the class. Allows the property to be called directly from the class without requiring a throw-away instance. @abc.abstractattribute - a simple, non-callable variable that must be overridden in subclasses @abc.abstractstaticproperty - like @abc.abstractproperty only for @staticproperty @abc.abstractclassproperty - like @abc.abstractproperty only for @classproperty --rich

10 15

Specify number of items to allocate for array.array() constructor
by Sven Rahmann 21 Feb '20

21 Feb '20

At the moment, the array module of the standard library allows to create arrays of different numeric types and to initialize them from an iterable (eg, another array). What's missing is the possiblity to specify the final size of the array (number of items), especially for large arrays. I'm thinking of suffix arrays (a text indexing data structure) for large texts, eg the human genome and its reverse complement (about 6 billion characters from the alphabet ACGT). The suffix array is a long int array of the same size (8 bytes per number, so it occupies about 48 GB memory). At the moment I am extending an array in chunks of several million items at a time at a time, which is slow and not elegant. The function below also initializes each item in the array to a given value (0 by default). Is there a reason why there the array.array constructor does not allow to simply specify the number of items that should be allocated? (I do not really care about the contents.) Would this be a worthwhile addition to / modification of the array module? My suggestions is to modify array generation in such a way that you could pass an iterator (as now) as second argument, but if you pass a single integer value, it should be treated as the number of items to allocate. Here is my current workaround (which is slow): def filled_array(typecode, n, value=0, bsize=(1<<22)): """returns a new array with given typecode (eg, "l" for long int, as in the array module) with n entries, initialized to the given value (default 0) """ a = array.array(typecode, [value]*bsize) x = array.array(typecode) r = n while r >= bsize: x.extend(a) r -= bsize x.extend([value]*r) return x

14 20

i18n and Python tracebacks
by Andre Roberge 28 Oct '12

28 Oct '12

I think it would be a good idea if Python tracebacks could be translated into languages other than English - and it would set a good example. For example, using French as my default local language, instead of >>> 1/0 Traceback (most recent call last): File "<stdin>", line 1, in <module> ZeroDivisionError: integer division or modulo by zero I might get something like >>> 1/0 Suivi d'erreur (appel le plus récent en dernier) : Fichier "<stdin>", à la ligne 1, dans <module> ZeroDivisionError: division entière ou modulo par zéro André

9 15

Cofunctions PEP - Revision 4
by Greg Ewing 13 Oct '12

13 Oct '12

Here's an updated version of the PEP reflecting my recent suggestions on how to eliminate 'codef'. PEP: XXX Title: Cofunctions Version: $Revision$ Last-Modified: $Date$ Author: Gregory Ewing <greg.ewing(a)canterbury.ac.nz> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 13-Feb-2009 Python-Version: 3.x Post-History: Abstract ======== A syntax is proposed for defining and calling a special type of generator called a 'cofunction'. It is designed to provide a streamlined way of writing generator-based coroutines, and allow the early detection of certain kinds of error that are easily made when writing such code, which otherwise tend to cause hard-to-diagnose symptoms. This proposal builds on the 'yield from' mechanism described in PEP 380, and describes some of the semantics of cofunctions in terms of it. However, it would be possible to define and implement cofunctions independently of PEP 380 if so desired. Specification ============= Cofunction definitions ---------------------- A cofunction is a special kind of generator, distinguished by the presence of the keyword ``cocall`` (defined below) at least once in its body. It may also contain ``yield`` and/or ``yield from`` expressions, which behave as they do in other generators. From the outside, the distinguishing feature of a cofunction is that it cannot be called the same way as an ordinary function. An exception is raised if an ordinary call to a cofunction is attempted. Cocalls ------- Calls from one cofunction to another are made by marking the call with a new keyword ``cocall``. The expression :: cocall f(*args, **kwds) is evaluated by first checking whether the object ``f`` implements a ``__cocall__`` method. If it does, the cocall expression is equivalent to :: yield from f.__cocall__(*args, **kwds) except that the object returned by __cocall__ is expected to be an iterator, so the step of calling iter() on it is skipped. If ``f`` does not have a ``__cocall__`` method, or the ``__cocall__`` method returns ``NotImplemented``, then the cocall expression is treated as an ordinary call, and the ``__call__`` method of ``f`` is invoked. Objects which implement __cocall__ are expected to return an object obeying the iterator protocol. Cofunctions respond to __cocall__ the same way as ordinary generator functions respond to __call__, i.e. by returning a generator-iterator. Certain objects that wrap other callable objects, notably bound methods, will be given __cocall__ implementations that delegate to the underlying object. Grammar ------- The full syntax of a cocall expression is described by the following grammar lines: :: atom: cocall | <existing alternatives for atom> cocall: 'cocall' atom cotrailer* '(' [arglist] ')' cotrailer: '[' subscriptlist ']' | '.' NAME Note that this syntax allows cocalls to methods and elements of sequences or mappings to be expressed naturally. For example, the following are valid: :: y = cocall self.foo(x) y = cocall funcdict[key](x) y = cocall a.b.c[i].d(x) Also note that the final calling parentheses are mandatory, so that for example the following is invalid syntax: :: y = cocall f # INVALID New builtins, attributes and C API functions -------------------------------------------- To facilitate interfacing cofunctions with non-coroutine code, there will be a built-in function ``costart`` whose definition is equivalent to :: def costart(obj, *args, **kwds): try: m = obj.__cocall__ except AttributeError: result = NotImplemented else: result = m(*args, **kwds) if result is NotImplemented: raise TypeError("Object does not support cocall") return result There will also be a corresponding C API function :: PyObject *PyObject_CoCall(PyObject *obj, PyObject *args, PyObject *kwds) It is left unspecified for now whether a cofunction is a distinct type of object or, like a generator function, is simply a specially-marked function instance. If the latter, a read-only boolean attribute ``__iscofunction__`` should be provided to allow testing whether a given function object is a cofunction. Motivation and Rationale ======================== The ``yield from`` syntax is reasonably self-explanatory when used for the purpose of delegating part of the work of a generator to another function. It can also be used to good effect in the implementation of generator-based coroutines, but it reads somewhat awkwardly when used for that purpose, and tends to obscure the true intent of the code. Furthermore, using generators as coroutines is somewhat error-prone. If one forgets to use ``yield from`` when it should have been used, or uses it when it shouldn't have, the symptoms that result can be extremely obscure and confusing. Finally, sometimes there is a need for a function to be a coroutine even though it does not yield anything, and in these cases it is necessary to resort to kludges such as ``if 0: yield`` to force it to be a generator. The ``cocall`` construct address the first issue by making the syntax directly reflect the intent, that is, that the function being called forms part of a coroutine. The second issue is addressed by making it impossible to mix coroutine and non-coroutine code in ways that don't make sense. If the rules are violated, an exception is raised that points out exactly what and where the problem is. Lastly, the need for dummy yields is eliminated by making it possible for a cofunction to call both cofunctions and ordinary functions with the same syntax, so that an ordinary function can be used in place of a cofunction that yields zero times. Record of Discussion ==================== An earlier version of this proposal required a special keyword ``codef`` to be used in place of ``def`` when defining a cofunction, and disallowed calling an ordinary function using ``cocall``. However, it became evident that these features were not necessary, and the ``codef`` keyword was dropped in the interests of minimising the number of new keywords required. The use of a decorator instead of ``codef`` was also suggested, but the current proposal makes this unnecessary as well. It has been questioned whether some combination of decorators and functions could be used instead of a dedicated ``cocall`` syntax. While this might be possible, to achieve equivalent error-detecting power it would be necessary to write cofunction calls as something like :: yield from cocall(f)(args) making them even more verbose and inelegant than an unadorned ``yield from``. It is also not clear whether it is possible to achieve all of the benefits of the cocall syntax using this kind of approach. Prototype Implementation ======================== An implementation of an earlier version of this proposal in the form of patches to Python 3.1.2 can be found here: http://www.cosc.canterbury.ac.nz/greg.ewing/python/generators/cofunctions.h… If this version of the proposal is received favourably, the implementation will be updated to match. Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

11 29

change NoneType, NotImplementedType, & ellipses to return the appropriate singleton
by Ethan Furman 07 Aug '11

07 Aug '11

Consider: Python 3.2 (r32:88445, Feb 20 2011, 21:29:02) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. --> for ins in ({0:'0'}, (1,), set([2, 3]), [4, 5], 6, 'seven', ... 8.0, True, None): ... print(type(ins)) ... type(ins)() ... <class 'dict'> {} <class 'tuple'> () <class 'set'> set() <class 'list'> [] <class 'int'> 0 <class 'str'> '' <class 'float'> 0.0 <class 'bool'> False <class 'NoneType'> Traceback (most recent call last): File "<stdin>", line 3, in <module> TypeError: cannot create 'NoneType' instances I hit this issue when I had a dictionary of types to instantiate, one of which was NoneType. Is there any reason why we shouldn't change these three to produce the singleton instance instead of raising an error? Pro: They would then be consistent with every other built-in which will return either a new instance (int, str, float, etc), or the singleton instance that already exists (bool, and sometimes int & str), as well as no longer needing the `lambda:None` workaround (or worse, a try-except if you don't know what types are coming in). Con: Might be code that relies on the exception being raised -- is anybody aware of any? ~Ethan~

15 20

Make compile('1\n2\n', '', 'single') raise an exception instead of silently truncating?
by Devin Jeanpierre 06 Aug '11

06 Aug '11

compile('1\n2\n', '','single') == compile('1\n', '','single'). That is, it ignores the second statement ('2\n'), without offering a way for the caller to detect this. Considering that 'single' is primarily used to emulate the behaviour of the Python interpreter, most of the time, giving it multiple statements is an impossibility, and so that case doesn't matter and could raise an exception without affecting existing code. For example, the code module meets this description, as do debuggers and such. However, in cases where it _is_ possible to give the compiler multiple statements, the user should be warned that his input isn't valid, somehow. For example, the following doctest will mysteriously fail, because it was written incorrectly (doctest uses 'single'): >>> import sys ... sys.stdout.write('foo\n') foo This is because the second statement in the doctest was silently discarded by compile(). It might not always be clear to users how to fix this, and I think this kind of non-obvious error would exist in any use of 'single' that can in theory involve multiple statements, through user error or program bug. So I'd appreciate it if compile() raised an exception in this case. Perhaps SyntaxError or ValueError. Devin

2 4

New imaplib implementation for Python 3.2+ standard library
by Maxim Khitrov 04 Aug '11

04 Aug '11

Hello python-ideas, My most recent project lead me down a path that eventually ended up at a new implementation of imaplib based on [RFC-3501]. Although I started the project by gradually adding functionality to the existing IMAP4 library, some of the features that I required simply could not be merged in (without breaking everything). As a result, I wrote my own version of the library, which incorporates all existing functionality of imaplib and includes many of my own improvements. I wrote a PEP-style readme file that describes all the details of why the library was written and how it works, which is available from my mercurial repository: http://hg.mxcrypt.com/python/imaplib2/raw-file/tip/README The same repository also contains the library code and an example script that you can run if you have access to an IMAP4 server: http://hg.mxcrypt.com/python/imaplib2/ Is there any interest in adding my code to a future version of Python 3.x standard library? - Max

7 19

anonymous object support
by Herman Sheremetyev 04 Aug '11

04 Aug '11

It is currently somewhat difficult/awkward to create arbitrary "anonymous" objects in Python. For example, to make some object that simply has a foo() method you have to declare a class that defines foo() and instantiate it: class Foo: def foo(self, x): return x obj = Foo() It would be a bit easier if Python allowed attributes to be set on instances of builtins the way it does for user-defined classes. This would at least allow something like: obj = object() obj.foo = lambda x: x Alas, that is not possible either as the assignment raises an AttributeError when dealing with instances of builtins. I would like to propose a different solution: alter object.__new__() to support keyword arguments and set the attributes named in the keyword arguments to their requested value. This would get us code like: obj = object(foo=1, bar=lambda x: x) obj.foo >>> 1 obj.bar(2) >>> 2 Python implementation of __new__ would look something like the following, though I suppose this would need to be done at the C level in reality: def __new__(typ, **kwargs): obj = object.__new__(typ) for attr, value in kwargs.items(): setattr(obj, attr, value) return obj So why do we need this? Primarily, this sort of thing is very common in tests where it is often useful to generate anonymous objects with default values. As a result, a number of libraries have risen to fill this need which is a pretty strong indicator that it is a useful pattern, and everyone using one of a dozen different libraries to accomplish the same basic task does not seem very Pythonic. I could also see this being quite useful in quick prototyping scenarios in order to avoid using dicts when an object is more suitable without requiring the overhead of defining a class. I'm curious if people on this list would be interested in seeing support for this added to the language, or if there are some good reasons I have overlooked for why this sort of change might be undesirable. Cheers, -Herman

14 23

+1, +0.5
by Christopher King 30 Jul '11

30 Jul '11

What does +1 or +0.5 mean. I assume that they are some sort of rating, so how does the rating system work. Thanks

2 1

Enums
by Guido van Rossum 29 Jul '11

29 Jul '11

On Sun, Jul 24, 2011 at 3:47 PM, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > We've actually been down the 'namespace object' road years ago (Steve > Bethard even wrote a proto-PEP IIRC) and it suffered the same fate as > most enum PEPs: everyone has slightly different ideas on what should > be supported, so you end up being faced with one of two options: > 1. Ignore some of the use cases (so some users still have to do their own thing) > 2. Support all of the use cases by increasing the API complexity (so > many users will still do their own thing instead of learning the new > API) For enums, I think we should just pick a solution. I'm in favor of Barry Warsaw's version, flufl.enum. -- --Guido van Rossum (python.org/~guido)

29 100