Mailman 3 November 2010 - Python-ideas

i18n and Python tracebacks
by Andre Roberge Oct. 28, 2012

Oct. 28, 2012

I think it would be a good idea if Python tracebacks could be translated into languages other than English - and it would set a good example. For example, using French as my default local language, instead of >>> 1/0 Traceback (most recent call last): File "<stdin>", line 1, in <module> ZeroDivisionError: integer division or modulo by zero I might get something like >>> 1/0 Suivi d'erreur (appel le plus récent en dernier) : Fichier "<stdin>", à la … [View More]

9 15

Cofunctions PEP - Revision 4
by Greg Ewing Oct. 13, 2012

Oct. 13, 2012

Here's an updated version of the PEP reflecting my recent suggestions on how to eliminate 'codef'. PEP: XXX Title: Cofunctions Version: $Revision$ Last-Modified: $Date$ Author: Gregory Ewing <greg.ewing(a)canterbury.ac.nz> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 13-Feb-2009 Python-Version: 3.x Post-History: Abstract ======== A syntax is proposed for defining and calling a special type of generator called a 'cofunction'. It is designed to provide a … [View More]streamlined way of writing generator-based coroutines, and allow the early detection of certain kinds of error that are easily made when writing such code, which otherwise tend to cause hard-to-diagnose symptoms. This proposal builds on the 'yield from' mechanism described in PEP 380, and describes some of the semantics of cofunctions in terms of it. However, it would be possible to define and implement cofunctions independently of PEP 380 if so desired. Specification ============= Cofunction definitions ---------------------- A cofunction is a special kind of generator, distinguished by the presence of the keyword ``cocall`` (defined below) at least once in its body. It may also contain ``yield`` and/or ``yield from`` expressions, which behave as they do in other generators. From the outside, the distinguishing feature of a cofunction is that it cannot be called the same way as an ordinary function. An exception is raised if an ordinary call to a cofunction is attempted. Cocalls ------- Calls from one cofunction to another are made by marking the call with a new keyword ``cocall``. The expression :: cocall f(*args, **kwds) is evaluated by first checking whether the object ``f`` implements a ``__cocall__`` method. If it does, the cocall expression is equivalent to :: yield from f.__cocall__(*args, **kwds) except that the object returned by __cocall__ is expected to be an iterator, so the step of calling iter() on it is skipped. If ``f`` does not have a ``__cocall__`` method, or the ``__cocall__`` method returns ``NotImplemented``, then the cocall expression is treated as an ordinary call, and the ``__call__`` method of ``f`` is invoked. Objects which implement __cocall__ are expected to return an object obeying the iterator protocol. Cofunctions respond to __cocall__ the same way as ordinary generator functions respond to __call__, i.e. by returning a generator-iterator. Certain objects that wrap other callable objects, notably bound methods, will be given __cocall__ implementations that delegate to the underlying object. Grammar ------- The full syntax of a cocall expression is described by the following grammar lines: :: atom: cocall | <existing alternatives for atom> cocall: 'cocall' atom cotrailer* '(' [arglist] ')' cotrailer: '[' subscriptlist ']' | '.' NAME Note that this syntax allows cocalls to methods and elements of sequences or mappings to be expressed naturally. For example, the following are valid: :: y = cocall self.foo(x) y = cocall funcdict[key](x) y = cocall a.b.c[i].d(x) Also note that the final calling parentheses are mandatory, so that for example the following is invalid syntax: :: y = cocall f # INVALID New builtins, attributes and C API functions -------------------------------------------- To facilitate interfacing cofunctions with non-coroutine code, there will be a built-in function ``costart`` whose definition is equivalent to :: def costart(obj, *args, **kwds): try: m = obj.__cocall__ except AttributeError: result = NotImplemented else: result = m(*args, **kwds) if result is NotImplemented: raise TypeError("Object does not support cocall") return result There will also be a corresponding C API function :: PyObject *PyObject_CoCall(PyObject *obj, PyObject *args, PyObject *kwds) It is left unspecified for now whether a cofunction is a distinct type of object or, like a generator function, is simply a specially-marked function instance. If the latter, a read-only boolean attribute ``__iscofunction__`` should be provided to allow testing whether a given function object is a cofunction. Motivation and Rationale ======================== The ``yield from`` syntax is reasonably self-explanatory when used for the purpose of delegating part of the work of a generator to another function. It can also be used to good effect in the implementation of generator-based coroutines, but it reads somewhat awkwardly when used for that purpose, and tends to obscure the true intent of the code. Furthermore, using generators as coroutines is somewhat error-prone. If one forgets to use ``yield from`` when it should have been used, or uses it when it shouldn't have, the symptoms that result can be extremely obscure and confusing. Finally, sometimes there is a need for a function to be a coroutine even though it does not yield anything, and in these cases it is necessary to resort to kludges such as ``if 0: yield`` to force it to be a generator. The ``cocall`` construct address the first issue by making the syntax directly reflect the intent, that is, that the function being called forms part of a coroutine. The second issue is addressed by making it impossible to mix coroutine and non-coroutine code in ways that don't make sense. If the rules are violated, an exception is raised that points out exactly what and where the problem is. Lastly, the need for dummy yields is eliminated by making it possible for a cofunction to call both cofunctions and ordinary functions with the same syntax, so that an ordinary function can be used in place of a cofunction that yields zero times. Record of Discussion ==================== An earlier version of this proposal required a special keyword ``codef`` to be used in place of ``def`` when defining a cofunction, and disallowed calling an ordinary function using ``cocall``. However, it became evident that these features were not necessary, and the ``codef`` keyword was dropped in the interests of minimising the number of new keywords required. The use of a decorator instead of ``codef`` was also suggested, but the current proposal makes this unnecessary as well. It has been questioned whether some combination of decorators and functions could be used instead of a dedicated ``cocall`` syntax. While this might be possible, to achieve equivalent error-detecting power it would be necessary to write cofunction calls as something like :: yield from cocall(f)(args) making them even more verbose and inelegant than an unadorned ``yield from``. It is also not clear whether it is possible to achieve all of the benefits of the cocall syntax using this kind of approach. Prototype Implementation ======================== An implementation of an earlier version of this proposal in the form of patches to Python 3.1.2 can be found here: http://www.cosc.canterbury.ac.nz/greg.ewing/python/generators/cofunctions.h… If this version of the proposal is received favourably, the implementation will be updated to match. Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: [View Less]

11 29

Bring back callable()
by Antoine Pitrou March 7, 2011

March 7, 2011

Hello, Python 3 has removed callable() under the justification that's it's not very useful and duck typing (EAFP) should be used instead. However, it has since been felt by many people that it was an annoying loss; there are situations where you truly want to know whether something is a callable without actually calling it (for example when writing sophisticated decorators, or simply when you want to inform the user of an API misuse). The substitute of writing `isinstance(x, collections.… [View More]

17 21

Possible PEP 380 tweak
by Guido van Rossum Dec. 11, 2010

Dec. 11, 2010

[Changed subject] > On 2010-10-25 04:37, Guido van Rossum wrote: >> This should not require threads. >> >> Here's a bare-bones sketch using generators: [...] On Mon, Oct 25, 2010 at 3:19 AM, Jacob Holm <jh(a)improva.dk> wrote: > If you don't care about allowing the funcs to raise StopIteration, this > can actually be simplified to: [...] Indeed, I realized this after posting. :-) I had several other ideas for improvements, e.g. being able to pass an initial … [View More]value to the reduce-like function or even being able to supply a reduce-like function of one's own. > More interesting (to me at least) is that this is an excellent example > of why I would like to see a version of PEP380 where "close" on a > generator can return a value (AFAICT the version of PEP380 on > http://www.python.org/dev/peps/pep-0380 is not up-to-date and does not > mention this possibility, or even link to the heated discussion we had > on python-ideas around march/april 2009). Can you dig up the link here? I recall that discussion but I don't recall a clear conclusion coming from it -- just heated debate. Based on my example I have to agree that returning a value from close() would be nice. There is a little detail, how multiple arguments to StopIteration should be interpreted, but that's not so important if it's being raised by a return statement. > Assuming that "close" on a reduce_collector generator instance returns > the value of the StopIteration raised by the "return" statements, we can > simplify the code even further: > > > def reduce_collector(func): > try: > outcome = yield > except GeneratorExit: > return None > while True: > try: > val = yield > except GeneratorExit: > return outcome > outcome = func(outcome, val) > > def parallel_reduce(iterable, funcs): > collectors = [reduce_collector(func) for func in funcs] > for coll in collectors: > next(coll) > for val in iterable: > for coll in collectors: > coll.send(val) > return [coll.close() for coll in collectors] > > > Yes, this is only saving a few lines, but I find it *much* more readable... I totally agree that not having to call throw() and catch whatever it bounces back is much nicer. (Now I wish there was a way to avoid the "try..except GeneratorExit" construct in the generator, but I think I should stop while I'm ahead. :-) The interesting thing is that I've been dealing with generators used as coroutines or tasks intensely on and off since July, and I haven't had a single need for any of the three patterns that this example happened to demonstrate: - the need to "prime" the generator in a separate step - throwing and catching GeneratorExit - getting a value from close() (I did have a lot of use for send(), throw(), and extracting a value from StopIteration.) In my context, generators are used to emulate concurrently running tasks, and "yield" is always used to mean "block until this piece of async I/O is complete, and wake me up with the result". This is similar to the "classic" trampoline code found in PEP 342. In fact, when I wrote the example for this thread, I fumbled a bit because the use of generators there is different than I had been using them (though it was no doubt thanks to having worked with them intensely that I came up with the example quickly). So, it is clear that generators are extremely versatile, and PEP 380 deserves several good use cases to explain all the API subtleties. BTW, while I have you, what do you think of Greg's "cofunctions" proposal? -- --Guido van Rossum (python.org/~guido) [View Less]

7 50

Detecting circular imports ?
by Tarek Ziadé Dec. 6, 2010

Dec. 6, 2010

Hello, This morning I tried to fix an issue for a while before I realized I had a circular import. This issue is not obvious because you get a cryptic error, like an AttributeError and it can tak a while before finding out. I don't know of this was mentioned before, or how hard it would be. But it would be nice if Python had a specific "CircularImportError" raised in that case, or something.. That would be a fabulous hint for developers. Cheers Tarek -- Tarek Ziadé | http://ziade.org

12 17

`issubclass` shouldn't be raising exceptions for non-type inputs
by cool-RR Dec. 5, 2010

Dec. 5, 2010

`issubclass(1, list)` raises an Exception, complaining that `1` is not a class. This is wrong in my opinion. It should just return False. Use case: I have an object which can be either a list, or a string, or a callable, or a type. And I want to check whether it's a sub-class of some base class. So I don't think I should be taking extra precautions before using `issubclass`: If my object is not a subclass of the given base class, I should just get `False`. Ram.

8 14

Adding `Unpicklable` to the `collections` module
by cool-RR Dec. 4, 2010

Dec. 4, 2010

Hello. Recently I had the need to filter objects based on whether they're picklable or not: http://stackoverflow.com/questions/4080688/python-pickling-a-dict-with-some… I'm not sure what's a good way to check for a specific object whether it's picklable. <http://stackoverflow.com/questions/4080688/python-pickling-a-dict-with-some…>This led me to think: Maybe we should have an `Unpicklable` abstract base class in the `collections` module? Then various unpicklable classes, like locks, … [View More]

19 38

Truly international Python
by Dima Tisnek Dec. 2, 2010

Dec. 2, 2010

Let's recall Guido's old Computer Programming for Everybody (CP4E) proposal. Nowadays that Python is established, it's high time to push Python into education, especially first programming language education. I think, in the modern world it means pre-school. Now the larger part of the world's children doesn't learn English before school, therefore we need to have truly localized Python. Some might recall a Python derivative demo with unicode variable names (link anyone?). I think we ought … [View More]to go further. For example, consider imaginary language pig latin: """This does that""" --> """Thiso acto thato""" # docstrings __version__ = (1,2,3) --> __versio__ = (1,2,3) # variable names import time --> importo chrono # standard module names def foo(): pass --> defo foo(): passo # Python keywords "foo".upper() --> "foo".uppero() # standard library raise Xx("undefined") --> raisio Xx("indifinito") # errors #!/usr/bin/python --> #!/usr/bin/pythono # executable name #!/usr/bin/python --> #!/usero/binaro/pythono # name and path Of course there are concerns for many languages: Each language needs to establish stable translations for keywords, basic types, standard modules, methods in standard modules, etc. Some languages don't support word spaces natively Some languages have different punctuation rules, e.g. comma for decimal point Some languages use different quotes RTL languages spell words RTL yet (some/all?) spell numbers LTR Hopefully none has to recreate 10,000-separator system ;-) Anyhow, it's not the issue of core Python to support particular languages, what is needed is: the concept that this is needed, and the base where from a particular localization can evolve from Here, a fun example, how Python might look like in google-translate-simplified-chinese. Blame google, not me as I know very little about this language. """This does that""" --> """这是""" # docstrings __version__ = (1,2,3) --> __版本__ = (1,2,3) # variable names import time --> 进口时间 # standard module names def foo(): pass --> 业美孚（）：通过 # Python keywords "foo".upper() --> “富” 上层（） # standard library raise Xx("undefined") --> 提高。二十（“未定义”） # errors #!/usr/bin/python --> #!/usr/bin/蛇 # executable name #!/usr/bin/python --> #!/用户/二进制/蛇 # name and path I track this here and will update with the received feedback: http://pythonic-wisdom.blogspot.com/2010/11/truly-international-python.html [View Less]

8 10

In-process interpreters
by Dima Tisnek Nov. 30, 2010

Nov. 30, 2010

Perhaps you can clarify what exactly you want to do. I can see at least 2 distict cases 1. Multithreaded web server (or even browser) * interpreters need separate imports at least pure python modules should be loaded and unloaded, different versions of same python modules could be used by different interpreters if different versions of C extensions are needed, different dynamic loading might be needed btw, is it still the case that C extensions cannot be unloaded? * interpreters need … [View More]

1 0

gc callbacks
by Kristján Valur Jónsson Nov. 29, 2010

Nov. 29, 2010

Hi there. Some months back I mentioned that I had other stuff in store for GC. Here is an idea for you. This particular idea is just a generalization of a system we've used in EVE for years now: garbage collection callbacks. The need, originally, was to be able to quantify the time spent in garbage collection runs. Since they occur out of direct control of the application, we needed to have python tell us about it somehow. We added gc.register_callback(), a function that added a callable to … [View More]

3 4

compile() getting an optmize flag
by Kristján Valur Jónsson Nov. 27, 2010

Nov. 27, 2010

When using python to compile to bytecode, it is really inconvenient that the compiler is fixed into using whatever opmiziation level the interpreter was started with. This makes it impossible to create e.g. .zip libraries of different optimization levels from a single python session. What is more, a program that creates optimized .zip files cannot itself have active asserts and so on. Adding the optimize state to the compiler´s state outht not to be too difficult. We could then: compile(… [View More]

4 3

syntax for set
by Alexandre Conrad Nov. 26, 2010

Nov. 26, 2010

Hi, I am sure this has come up before and if it doesn't exist I believe there is a good reason for it. Anyway, here's my thought: how about having a syntax representation for sets? A mix of a dict and list/tuple syntax would make the most sense, such as: >>> isinstance({"foo", "bar"}, set) True The problem here is: how to represent an empty set as {} would conflict with the dict repr. Then how about using a brand new "container" syntax that wouldn't conflict with existing ones? … [View More]

25 54

Re: [Python-ideas] syntax for set
by Bruce Leban Nov. 25, 2010

Nov. 25, 2010

> The idea is flawed. No, really! People would start > writing integer literals as floats. And the scheme > doesn't cover floats (there is no standard way of > writing decimal places in Roman numerals AFAIK). > Rob Cliffe 3.14 == CCCXIVE-II (unscientific notation) I've always thought COBOL did a better job of EIBTI with spelled out keywords like ADD instead of + etc. This would clearly one up COBOL. (ADD ONE TO COBOL GIVING PYTHON) --- Bruce (via android)

1 0

Re: [Python-ideas] Set Syntax
by Gerald Britton Nov. 24, 2010

Nov. 24, 2010

Hi -- thought I'd chime in on this one: > I'd be happy with: > > * {:} for empty dict() (as a collection of key-value *pairs*) > * {.} for empty set() (as a similar collection of *single* elements) > > And {} for empty dict() as well -- to keep compatibility (maybe to be > deprecated later). Note that dictionaries and sets have clear() methods: >>> d = {1:2} >>> d {1: 2} >>> d.clear() >>> d {} >>> s = set([1,2]) >>> s … [View More]

9 9

Adding an optional function argument to all() and any() builtins
by Andy Buckley Nov. 22, 2010

Nov. 22, 2010

I may be missing a standard idiom, but it seems to me that the any(...) and all(...) builtins are unnecessarily limited by requiring that the iterable they take as an argument is already in a form suitable for the intended kind of boolean comparison. Most of the time, when I want to check that any or all of a collection matches some test criterion, my iterable is not already in a valid form to pass to any() or all(), and so I either have to explicitly re-code a slightly modified version of the … [View More]

17 26

syntax for set
by average Nov. 22, 2010

Nov. 22, 2010

On Thu, Nov 18, 2010 at 3:50 PM, Steven D'Aprano <steve(a)pearwood.info>wrote: > average wrote: > > Truly. But the point is that the universe will inevitably evolve towards >> elegance. >> > > "Inevitably"? I think the only think we can say that the universe will > inevitably evolve towards is a state of maximum entropy. But frankly, I > don't see the relevance to the future heat death of the universe in hundreds > of billions of years to Python's … [View More]syntax. > Come on now :) I was speaking of the programmer's universe. But to the extent the universe is logical, it will also evolve (or has evolved?) towards elegance (that is, after all what's pushing the physicist's ideal of a Theory of Everything!). Which brings us to the mathematicians dilemma: do we create the universe or do we discover it? > To only "go on with the information it has and the users it has" > ensures that it remains merely a branch off the trunk of the ideal > (consequently drawing few new users), that eventually will need either to > be > *deliberately* pruned, or will wither and break off unexpectedly at some > future point. Remember, the bulk of new, loyal, users are drawn by an > ideal > (you might remember yourself being among them), not even necessarily by how > big the library is. > I beg to differ... it seems to me that users are drawn to languages for many > reasons, and while elegance can be one of them, it is fairly low down the > list. Why else would C++, VB and PHP be so popular, while Haskell and Scheme > remain tiny and unpopular (although very influential)? > Well, you certainly have the issue of momentum for one part of the explanation. Many are not willing do ditch their old toolset even when a better one is presented. The other is that the most elegant languages are sometimes those understood by the fewest.... > Besides, how do you define "elegant"? I've heard it said, in full > seriousness, that BrainF*** is the most elegant language because it is a > fully Turing-complete language in only six commands (plus two more for IO). > > I can define it quite succinctly: Komolgorov semantic density. Each element of syntax of a language can be seen as a condensing down to its Kolmolgorov ideal the tightest expression of a common programmatic need. Completeness is a necessary but insufficient condition (so BrainFuck is out of the question). Personally, I see Python as an extremely elegant language, but that elegance > is tempered by a very strong dose of practicality and realism. Gratuitously > breaking backwards compatibility just so that there is a literal for the > empty set is not what I call either practical or realistic. > The breaking of backwards compatibility is in no way gratuitous. It may be demolishing of a long-standing local-maxima, but hardly gratuitous. Just a few ideas, nothing I'm trying to make a religious war on... marcos [View Less]

2 1

In-process interpreters
by Swapnil Talekar Nov. 19, 2010

Nov. 19, 2010

I am working on a project where we have a very peculiar requirements for an embedded interpreter. In my opinion these requirements cannot be extremely rare. I am posting this message to get some feedback from people who have worked on similar stuff or are interested in such a solution. If such a solution would be useful to a lot people if readily available as open source. We need to run multiple interpreters in-process. Which means we need to create completely independent environments for each … [View More]

7 7

Fwd: In-process interpreters
by Swapnil Talekar Nov. 17, 2010

Nov. 17, 2010

> Em Wed, 17 Nov 2010 13:56:43 +0530 > Swapnil Talekar <swapnil.st(a)gmail.com> escreveu: > > > We need to run multiple interpreters in-process. Which means we need > > to create completely independent environments for each interpreter > > using some very dirty hack. Since Python inherently does not support > > this. Python threads( created using python's threading module ) are > > not of much help because of the GIL. > > For this purpose the … [View More]

1 0

pool threads
by Kristján Valur Jónsson Nov. 16, 2010

Nov. 16, 2010

Hi there. Occasionally you want to do something on a worker thread from Python. Some operating systems, like Windows, maintain a threadpool for which you can queue a user work item to execute. I was thinking that it could be a good idea to expose such a functionality by the thread module. With something like: def thread.queue_call(callable, args, kwargs): #invoke the OS threadpool api and have callable called Then in threading.py: Try: From thread import queue_call Except ImportError: … [View More]

5 5

Re: [Python-ideas] stats module Was: minmax() function ...
by Steven D'Aprano Nov. 14, 2010

Nov. 14, 2010

Seconding Raymond's 'drat'. Resending to python-ideas. On Sat, 16 Oct 2010 07:00:16 am Raymond Hettinger wrote: > Hello guys. If you don't mind, I would like to hijack your thread > :-) Please do :) > A few years ago, Guido and other python devvers supported a > proposal I made to create a stats module, but I didn't have time > to develop it. [...] > I think the creativity and energy of this group is much better > directed at building a quality stats module (perhaps … [View More]

6 12

PEP 3151 - Reworking the OS and IO exception hierarchy (again)
by Antoine Pitrou Nov. 12, 2010

Nov. 12, 2010

Hello, Georg (who manages the 3.2 release) suggested that I push PEP 3151 (*) in the hope that it can get in before the 3.2 feature freeze. Is there any consensus that PEP 3151 should get in 3.2, or even that it should get in at all? If there's some process issue that I'm missing, I'd be glad to know about it. (*) http://www.python.org/dev/peps/pep-3151/ Thanks Antoine.

4 13

time.wallclock() or other similar stuff
by Kristján Valur Jónsson Nov. 8, 2010

Nov. 8, 2010

Bringing this in from python-dev, and http://bugs.python.org/issue10278 Summary: time.clock() means two different things on Windows and Unix. So, if you need to measure time across a blocking call in a portable way, you need to use time.time() (this fact is not documented). Time.time has its own problems though, such as low resolution (compared to time.clock on windows) and being susceptible to being adjusted by the user. I proposed adding a time.wallclock() to address this issue, which … [View More]

17 25

Multi-line strings that respect indentation
by Daniel da Silva Nov. 7, 2010

Nov. 7, 2010

On several occasions I have run into code that will do something like the following with a multiline string: def some_func(): > x, y = process_something() > > val = """ > <xml> > <myThing> > <val>%s</val> > <otherVal>%s</otherVal> > </myThing> > </xml> > """ % (x, y) > > return val > To me, this is rather ugly because it messes up the indentation of some_func(). Suppose we could … [View More]

15 21

Remapping a table's keys
by Andrey Fedorov Nov. 5, 2010

Nov. 5, 2010

Hi guys, I often run into the problem when I need to systematically rename a bunch of keys in a table. For example, a JSON API returns an object where I need to map "Media Cost" to "adjusted_partner_cost", "Impressions" to "bid_count", etc. One way of doing this is: key_map = { 'Media Cost': 'adjusted_partner_cost', 'Impressions': 'bids_won', 'Day': 'date' } mykeys = {} for k, v in dict_from_elsewhere.items(): if k in key_map: ret[key_map[k]] = v else: ret[k] = … [View More]

3 2

Factory functions (was: RE: time.wallclock() or other similar stuff)
by Kristján Valur Jónsson Nov. 3, 2010

Nov. 3, 2010

Completely off topic. What's with factory functions? Why not a class? And why are there factory functions in threading.py and not classes? K -----Original Message----- From: python-ideas-bounces+kristjan=ccpgames.com(a)python.org [mailto:python-ideas-bounces+kristjan=ccpgames.com@python.org] On Behalf Of Greg Ewing Sent: Wednesday, November 03, 2010 6:19 To: python-ideas(a)python.org Subject: Re: [Python-ideas] time.wallclock() or other similar stuff However, it suggests a factory function … [View More]

5 7