Mailman 3 March 2010 - Python-Dev

Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously
by P.J. Eby March 7, 2010

March 7, 2010

At 08:39 AM 3/7/2010 -0800, Jeffrey Yasskin wrote: >On Sun, Mar 7, 2010 at 7:48 AM, P.J. Eby <pje(a)telecommunity.com> wrote: > > At 02:49 PM 3/7/2010 +1000, Nick Coghlan wrote: > >> > >> P.J. Eby wrote: > >> > (Personally, I think it would be better to just drop the ambitious title > >> > and scope, and go for the "nice task queue" scope. Â I > imagine, too, that > >> > in that case Jean-Paul wouldn't need to worry about … [View More]it being raised as a > >> > future objection to Deferreds or some such getting into the stdlib.) > >> > >> This may be a terminology thing - to me futures *are* just a nice way to > >> handle farming tasks out to worker threads or processes. You seem to see > >> them as something more comprehensive than that. > > > > Actual futures are, yes. Â Specifically, futures are a mechanism for > > asynchronous computation, whereas the PEP seems to be all about > > synchronously managing parallel tasks. Â That's a huge difference. > > > > Technically, the things in the PEP (and by extension, Java's futures) match > > the letter of the definition of a future, but not (IMO) the > spirit. Â There's > > no clean way to compose them, and at base they're more about parallelism > > than asynchrony. > >Do you have an example of a language or library that uses the term >"future" to refer to what you're talking about? I'm curious to see >what it looks like. The wikipedia page menetioned earlier in the discussion has several examples. Twisted's Deferreds are probably the best example of such a system in Python, though, as far as I know. The features proposed in the PEP are basically akin to Twisted's deferToThread(), i.e. "return a future for running this code in some other thread". However, you could easily make your own thread or process pool for executing tasks, using Deferreds as your "future" objects; deferToThread() is not particularly special, it's just one possible "executor", to use the PEP's terminology. Likewise, deferreds are not limited to representing parallel task execution - they can also represent asynchronous events such as incoming I/O or a waited-for mouse click. Deferreds allow composing calculation by stacking callbacks (in continuation-passing style), but they're most Pythonic (IMO) when used with a coroutine wrapping system like Eventlet or my earlier peak.events framework, so that you can use them in "yield" expressions. That way you can write synchronous-looking code that nonetheless executes asynchronously. Twisted uses Deferreds extensively, both for I/O and IPC, and of course for database access. However, the 'defer' module itself is conceptually distinct from both the I/O system, event loop, and threads: it's a largely-generic framework for managing asynchronous computation. Parallel task queueing is simply *one* possible application of Deferreds; their actual purpose is to provide a common API for working with values that aren't yet known. That is to say, futures. In contrast, the PEP manages only a very limited sort of "future" - the completion of a single task, which must be synchronously waited for. If one "future" in the PEP needs to wait for another, you tie up that thread or process in a busy loop... which rather limits how much parallelism you can have. Deferreds also allow you to write code that doesn't *care* whether the value has been determined or not... you can simply stack on what code you want to have run when the value is available, and it runs immediately if the value is there. Likewise, a deferred is not tied to the execution of a single piece of code - it's more of a rendezvous or synchronization point, that can be given a value to "fire" with. (In contrast, the PEP's "futures" are tightly coupled to a task execution model: they have a notion of running and being cancelled, for example.) For these reasons, I'd say that Deferreds are much more deserving of the title "futures", especially since there is no need to call a simple task queue a "futures" system. (Unless you're Java, I suppose, in which case you might want the name for marketing reasons. ;-) But in Python I assume we'd like to call a spade a spade, and a task queue a task queue.) [View Less]

2 1

[RELEASED] Python 2.7 alpha 4
by Benjamin Peterson March 6, 2010

March 6, 2010

On behalf of the Python development team, I'm overjoyed to announce the fourth alpha release of Python 2.7. Python 2.7 is scheduled (by Guido and Python-dev) to be the last major version in the 2.x series. Though more major releases have not been absolutely ruled out, it's likely that the 2.7 release will an extended period of maintenance for the 2.x series. 2.7 includes many features that were first released in Python 3.1. The faster io module, the new nested with statement syntax, … [View More]

1 0

doctest, unicode repr, and 2to3
by "Martin v. Löwis" March 5, 2010

March 5, 2010

Johan Harjano ran into an interesting problem when trying to run the Django test suite under Python 3.1. Django has doctests of the form >>> a6.headline u'Default headline' Even when converting the doctest with 2to3, the expected output is unmodified. However, in 3.x, the expected output will change (i.e. not produce an u"" prefix anymore). Now, it might be possible to reformulate the test case (e.g. use print() instead of relying on repr), however, this is undesirable as a) the … [View More]

11 13

Re: [Python-Dev] __file__
by Henning von Bargen March 4, 2010

March 4, 2010

Nick Coghlan wrote: > Another option is to remove bytecode-only support from the default > filesystem importer, but keep it for zipimport (since the stat call > savings don't apply in the latter case). +1 Baptiste Carvello wrote: > However, making a difference between zipimport and the filesystem importer means > the application will stop working if I unzip the library zip file, which is > surprising. > Unzipping the zip file can be handy when debugging a bug caused … [View More]

3 2

Release blocker: urllib2 issue 7540
by Fred Drake March 4, 2010

March 4, 2010

Regarding: http://bugs.python.org/issue7540 The change made in response to this issue report breaks existing uses of Python 2.6.4. The mechanize library frequently re-initializes the data in the request without re-using the request. This worked fine in the past, and does not trigger the problem reported without the request being re-used. Applications (including tests) that use mechanize now break with this TypeError. This includes much (if not all) of the Zope community, which uses … [View More]

2 1

code.python.org dead?
by Antoine Pitrou March 4, 2010

March 4, 2010

Hello, It seems someone has decided that code.python.org doesn't resolve anymore. This is annoying since there were Mercurial mirrors there ( at http://code.python.org/hg ). Until we make the official switch, these mirrors are still useful. Regards Antoine.

3 5

The fate of Distutils in Python 2.7
by Tarek Ziadé March 3, 2010

March 3, 2010

Hello, This is a follow-up of the Pycon summit + sprints on packaging. This is what we have planned to do: 1. refactor distutils in a new standalone version called distutils2 [this is done already and we are actively working in the code] 2. completely revert distutils in Lib/ and Doc/ so the code + doc is the same than the current 2.6.x branch 3. leave the new sysconfig module, that is used by the Makefile and the site module The rest of the work will happen in distutils2 and we will try to … [View More]

2 4

Re: [Python-Dev] __file__
by Barry Warsaw March 3, 2010

March 3, 2010

On Feb 03, 2010, at 12:42 PM, Brett Cannon wrote: >So what happens when only bytecode is present? We discussed this at Pycon and agreed that we will not support source-less deployments by default. The source file must exist or it will be an ImportError. This does not mean source-less deployments are not possible though. To support this use case, you'd have to write a custom import hook. We may write one as part of the PEP 3147 implementation. Contributions are of course welcome! -Barry

18 86

Caching function pointers in type objects
by Daniel Stutzbach March 3, 2010

March 3, 2010

In CPython, is it safe to cache function pointers that are in type objects? For example, if I know that some_type->tp_richcompare is non-NULL, and I call it (which may execute arbitrary user code), can I assume that some_type->tp_richcompare is still non-NULL? -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC <http://stutzbachenterprises.com>

4 8

FWD: Broken link in 2.6.5 rc 1 download page.
by Michael Foord March 3, 2010

March 3, 2010

As far as I can tell there is no Python 2.6.5 documentation online and the link to 3.1 docs is actually for 3.1.1 whilst the link for 3.1.1 docs is dead. The actual documentation doesn't seem to be in the pydotorg svn repository so I guess this a release manager issue? Michael Foord -------- Original Message -------- Subject: [Pydotorg] FWD: Broken link in 2.6.5 rc 1 download page. Date: Wed, 3 Mar 2010 08:45:36 -0800 From: Aahz <aahz(a)pythoncraft.com> Organization: The Cat &… [View More]amp; Dragon To: pydotorg(a)python.org ----- Forwarded message from Simon de Vlieger<simon(a)ikanobori.jp> ----- > From: Simon de Vlieger<simon(a)ikanobori.jp> > To: webmaster(a)python.org > Subject: Broken link in 2.6.5 rc 1 download page. > Date: Wed, 3 Mar 2010 09:57:29 +0100 > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Dear webmaster (I know no name), > > on this page: http://www.python.org/download/releases/2.6.5/ both links > for the documentation are incorrect. On this page: > http://www.python.org/doc/versions/ the link for the Python 3.1.1 is > broken. > > Hope to have informed you well enough. > > Regards, > > Simon de Vlieger > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (Darwin) > > iQIcBAEBAgAGBQJLjiR5AAoJEBBSHP7i+JXfidsP/0fLCK0sn5+mj9CPsobDant7 > 1tkXK7pwrbujXhpAyhjhE/kB5BhNI/h9XpmmUkLSZzz7AoOpE6KzlqCzOmwBmdAk > 6LxO2c4ZojICVB5zgNYXibDtnSwbBhcefbEsN6BSh1kI/3ThcIdWRYFavBAlxj1T > 6P7y8MOHUn3INLEw8foxV20geivcqzmMhRtfglMVs9KcB/PZWrK8fWw8K2Ov0B+U > ihxaywfiJWcdkFKTTLNvQCbVr6ZGke728/7/lcWLvdPlwRPfVteKlLDglWX0beBy > tgQyxmtxFx+AeCiLeyybnxEnxRVgU1nYxsdWluJuwFT4l+/BsJG3CkM522rQ3MWZ > +DYB6xfONlvfCSXaPn3fOYesEvXek9BxKLiXpjWlJsmXvDcdFN2sr24aR1cJEXL8 > ZRkiJO5VFzngIxytk0Oy/SeMkxCUtsq4TMlMT2OyxblxxRoLikPSsfgKRuDkQ5s+ > QK2yNxTgKs2yA5ux9kmZscmVgHcWqJFjkE72dJoARyHMra9Jvcr2U7GVE2zqcOCS > rObaqq61TpZmvlPUXlkLUueTUI9Djz01hAQUp+Vcwu8omVlIgZC0GbuXK8JDfQJd > 60WFI1BvUQg7mDB1ThUU6HUE9rF71921lx02YyZFzGFn1K2/w/bRhxrrzD5XwJDQ > fFwYKj+pwgHO5AU5hMCR > =rTPq > -----END PGP SIGNATURE----- ----- End forwarded message ----- -- Aahz (aahz(a)pythoncraft.com)<*> http://www.pythoncraft.com/ "Many customs in this life persist because they ease friction and promote productivity as a result of universal agreement, and whether they are precisely the optimal choices is much less important." --Henry Spencer _______________________________________________ Pydotorg mailing list Pydotorg(a)python.org http://mail.python.org/mailman/listinfo/pydotorg [View Less]

2 1