Mailman 3 March 2010 - Python-Dev

Bug tracker: no confirmation before deleting a message
by Victor Stinner March 11, 2010

March 11, 2010

Hi, I removed two messages by error in the bug tracker :-( The first message is msg100432 (first message of #8063) which has been restored by "admin". Thank you admin! The second message is msg98284, part of #7774. How can I restore it? Or can "admin" restore it please? :-) There is an open issue on Roundup tracker to add a button to restore a message or a deleted file: http://psf.upfronthosting.co.za/roundup/meta/issue267 I tried to following URL to restore the message, but it didn't work: http://bugs.python.org/issue8063?@action=edit&@add@messages=100432 Could you at least add a Javascript confirmation dialog please? :-) -- Victor Stinner http://www.haypocalc.com/

2 1

Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously
by Phillip J. Eby March 11, 2010

March 11, 2010

At 01:19 AM 3/6/2010, Jeffrey Yasskin wrote: >On Fri, Mar 5, 2010 at 10:11 PM, Phillip J. Eby <pje(a)telecommunity.com> wrote: > > I'm somewhat concerned that, as described, the proposed API ... > [creates] yet another alternative (and > > mutually incompatible) event loop system in the stdlib ... > >Futures are a blocking construct; they don't involve an event loop. And where they block is in a loop, waiting for events (completed promises) coming back from other threads or processes. The Motivation section of the PEP also stresses avoiding reinvention of such loops, and points to the complication of using more than one at a time as a justification for the mechanism. It seems relevant to at least address why wrapping multiprocessing and multithreading is appropriate, but *not* dealing with any other form of sync/async boundary, *or* composition of futures. On which subject, I might add, the PEP is silent on whether executors are reentrant to the called code. That is, can I call a piece of code that uses futures, using the futures API? How will the called code know what executor to use? Must I pass it one explicitly? Will that work across threads and processes, without explicit support from the API? IOW, as far as I can tell from the PEP, it doesn't look like you can compose futures without *global* knowledge of the application... and in and of itself, this seems to negate the PEP's own motivation to prevent duplication of parallel execution handling! That is, if I use code from module A and module B that both want to invoke tasks asynchronously, and I want to invoke A and B asynchronously, what happens? Based on the design of the API, it appears there is nothing you can do except refactor A and B to take an executor in a parameter, instead of creating their own. It seems therefore to me that either the proposal does not define its scope/motivation very well, or it is not well-equipped to address the problem it's setting out to solve. If it's meant to be something less ambitious -- more like a recipe or example -- it should properly motivate that scope. If it's intended to be a robust tool for composing different pieces of code, OTOH, it should absolutely address the issue of writing composable code... since, that seems to be what it says the purpose of the API is. (I.e., composing code to use a common waiting loop.) And, existing Python async APIs (such as Twisted's Deferreds) actually *address* this issue of composition; the PEP does not. Hence my comments about not looking at existing implementations for API and implementation guidance. (With respect to what the API needs, and how it needs to do it, not necessarily directly copying actual APIs or implementations. Certainly some of the Deferred API naming has a rather, um, "twisted" vocabulary.)

12 30

FWD: http://www.python.org/dev/patches/
by Aahz March 11, 2010

March 11, 2010

----- Forwarded message from Bob Vadnais <bob(a)boblicious.com> ----- > Date: Tue, 09 Mar 2010 00:37:33 -0500 > From: Bob Vadnais <bob(a)boblicious.com> > To: webmaster(a)python.org > Subject: http://www.python.org/dev/patches/ > > > Submit documentation patches the same way. When adding the patch, be > > sure to set the "Category" field to "Documentation". > > There doesn't appear to be a field named "Category". Perhaps there used > to be? The correct field name now appears to be "Components". ----- End forwarded message ----- -- Aahz (aahz(a)pythoncraft.com) <*> http://www.pythoncraft.com/ "Many customs in this life persist because they ease friction and promote productivity as a result of universal agreement, and whether they are precisely the optimal choices is much less important." --Henry Spencer

2 1

Python 2.6.5 release candidate 2 now available
by Barry Warsaw March 10, 2010

March 10, 2010

Hi everyone, The source tarballs and Windows installer for Python 2.6.5 release candidate 2 are now available: http://www.python.org/download/releases/2.6.5/ As usual, please download, install, and test them with your favorite projects and environments. A number of regressions and build problems on OS X have been fixed since rc1, and I'm really hoping we will not have to do an rc3. I'm currently planning on releasing 2.6.5 final on March 19, 2010. Enjoy, -Barry P.S. The Mac installer will hopefully be available soon.

1 0

argparse ugliness
by Neal Becker March 10, 2010

March 10, 2010

I generally enjoy argparse, but one thing I find rather ugly and unpythonic. parser.add_argument ('--plot', action='store_true') Specifying the argument 'action' as a string is IMO ugly.

17 29

[PEP 3148] futures - execute computations asynchronously
by Brian Quinlan March 10, 2010

March 10, 2010

Hi all, I recently submitted a daft PEP for a package designed to make it easier to execute Python functions asynchronously using threads and processes. It lets the user focus on their computational problem without having to build explicit thread/process pools and work queues. The package has been discussed on stdlib-sig but now I'd like this group's feedback. The PEP lives here: http://python.org/dev/peps/pep-3148/ Here are two examples to whet your appetites: """Determine if several numbers are prime.""" import futures import math PRIMES = [ 112272535095293, 112582705942171, 112272535095293, 115280095190773, 115797848077099, 1099726899285419] def is_prime(n): if n % 2 == 0: return False sqrt_n = int(math.floor(math.sqrt(n))) for i in range(3, sqrt_n + 1, 2): if n % i == 0: return False return True # Uses as many CPUs as your machine has. with futures.ProcessPoolExecutor() as executor: for number, is_prime in zip(PRIMES, executor.map(is_prime, PRIMES)): print('%d is prime: %s' % (number, is_prime)) """Print out the size of the home pages of various new sites (and Fox News).""" import futures import urllib.request URLS = ['http://www.foxnews.com/', 'http://www.cnn.com/', 'http://europe.wsj.com/', 'http://www.bbc.co.uk/', 'http://some-made-up-domain.com/'] def load_url(url, timeout): return urllib.request.urlopen(url, timeout=timeout).read() with futures.ThreadPoolExecutor(max_workers=5) as executor: # Create a future for each URL load. future_to_url = dict((executor.submit(load_url, url, 60), url) for url in URLS) # Iterate over the futures in the order that they complete. for future in futures.as_completed(future_to_url): url = future_to_url[future] if future.exception() is not None: print('%r generated an exception: %s' % (url, future.exception())) else: print('%r page is %d bytes' % (url, len(future.result()))) Cheers, Brian

20 51

Building thread-safe sqlite3 with Python 2.6.5rc1
by Darren Govoni March 9, 2010

March 9, 2010

Hi, I'm trying to get the thread-safe enabled for sqlite3 in my build of 2.6.5rc1, but its not taking and throws the old thread access error. I am running Ubuntu with other versions of Python, but I successfully built and installed sqlite 3.6.22, pysqlite 2.6.0. Do I have to set something explicit when building python 2.6.5 to find the updated sqlite3? thanks for any tips. Darren

3 5

unittest2 plus class and module level fixtures in unittest
by Michael Foord March 8, 2010

March 8, 2010

Hello all, unittest has seen quite a few new features since Python 2.6. For those of you who might have missed the announcements in other places, unittest2 is a backport of the new features in unittest to work with Python 2.4-2.6. It is already being used for the development of distutils2: http://pypi.python.org/pypi/unittest2/ <http://pypi.python.org/pypi/unittest2/> In other news, class and module fixtures (setUpClass / tearDownClass / setUpModule / tearDownModule) are now implemented in unittest (in trunk - not yet merged to py3k). These features are tested but I'm sure there are some lurking bugs or oddities, so please try them out. I have not yet added documentation for them; I'll pull it out from this email as a starting point. I'd rather this thread didn't become *another* debate on the merit of these features, but perhaps that is too much to hope for. Below are some notes on how class and module fixtures work. Class and module level fixtures are implemented in unittest.TestSuite. When the test suite encounters a test from a new class then tearDownClass from the previous class (if there is one) is called, followed by setUpClass from the new class. Similarly if a test is from a different module from the previous test then tearDownModule from the previous module is run, followed by setUpModule from the new module. After all the tests have run the final tearDownClass and tearDownModule are run. Note that shared fixtures do not play well with [potential] features like test parallelization and they break test isolation. They should be used with care. setUpClass and tearDownClass -------------------------------------- These must be implemented as class methods. import unittest class Test(unittest.TestCase): @classmethod def setUpClass(cls): cls._connection = createExpensiveConnectionObject() @classmethod def tearDownClass(cls): cls._connection.destroy() If you want the setUpClass and tearDownClass on base classes called then you must call up to them yourself. The implementations in unittest2.TestCase are empty. If an exception is raised during a setUpClass then the tests in the class are not run and the tearDownClass is not run. Skipped classes will not have setUpClass or tearDownClass run. A setUpClass that raises a unittest2.SkipTest exception will currently be reported as an error rather than a skip (although the effect is the same). This will be fixed at some point in the future. setUpModule and tearDownModule -------------------------------------------- These should be implemented as functions. def setUpModule(): createConnection() def tearDownModule(): closeConnection() If an exception is raised in a setUpModule then none of the tests in the module will be run and the tearDownModule will not be run. A setUpModule that raises a unittest.SkipTest exception will currently be reported as an error rather than a skip (although the effect is the same). This will be fixed at some point in the future. The Gory Details ---------------------- The default ordering of tests created by the unittest test loaders is to group all tests from the same modules and classes together. This will lead to setUpClass / setUpModule (etc) being called exactly once per class and module. If you randomize the order, so that tests from different modules and classes are adjacent to each other, then these shared fixture functions may be called multiple times in a single test run. This particular issue was a point of some disussion. Shared fixtures are not intended to work with suites with non-standard (incompatible) ordering (a BaseTestSuite still exists for frameworks that don't want to support shared fixtures), so this is already a pathological situation. There are a couple of things unittest could do in this situation: 1) Work out the best order to run the setUp and tearDowns, based on the position of the first and last test from each class / module in the suite. 2) Run setUp and tearDowns appropriately if consecutive tests are from different classes and modules. If your setUps do unrelated things then strategy 1 will be the best one, but (so long as your setups are repeatable) 2 will be fine just slower. If your setups do related things, for example pushing large test datasets into the same database, then having multiple setups active before the corresponding teardowns have been run will actually do the wrong thing and your tests will be broken. In addition strategy 1 relies on unrolling the test suite and introspecting all tests prior to the run, not all custom test suites implementations support this. Given that strategy 2 was substantially easier to implement, and less likely to cause broken test runs, that was what I went with. It is of course possible to add alternative strategies, plus the metadata to support them, if there proves to be a substantial need for this. One of the major use cases is for test order randomization. It would be easy to add a 'randomize' method to the suite that randomizes the order of test modules in a suite, the order of test classes within modules and the order of tests within a class. This would allow for tests to be run in a random order whilst remaining compatible with shared fixtures. If there are any exceptions raised during one of the shared fixture functions the test is reported as an error. Because there is no corresponding test instance an _ErrorHolder object (that has the same interface as a TestCase) is created to represent the error. If you are just using the standard unittest test runner then this detail doesn't matter, but if you are a framework author it may be relevant. I have a few minor features I'd like to add to unittest before 2.7 beta - standard out capturing, fail fast and ctrl-c handling (first ctrl-c will wait for current test to exit, second will force an exit - results will still be reported). After that the *really* big problem for unittest is the extensibility story. I'll be working on extensibility points for unittest that allow people to implement extensions without having to override functionality. This will take a fair bit of discussion and experimentation, so I will probably trial this in unittest2 first. It would be nice to have something in place for the release of 3.2, but that may be ambitious. All the best, Michael Foord -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies ("BOGUS AGREEMENTS") that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

3 3

Python 2.6.5 rc 2
by Barry Warsaw March 7, 2010

March 7, 2010

We are definitely going to need a 2.6.5 rc 2. We've had a number of critical issues fixed since rc1 and we have two more critical patches approved for landing which fix issues on OS X. I've given Ronald approval in the tracker to land these two patches. 8066 OS X installer: readline module breaks when targeting on 10.5 or 10.6 8067 OS X Installer: build errors on 10.6 when targeting 10.4 and earlier These are the only two release blockers left for 2.6.5 (at the moment). I would like to cut the rc2 release tomorrow, Monday 2010-03-08, so if anybody knows of any other bugs you think must go into 2.6.5, mark them release blockers now so that I can evaluate them. I really don't want there to be an rc3, and I'd still like to get 2.6.5 final out on Monday 2010-03-15. We'll see. -Barry

1 0

[RELEASED] Python 3.1.2 release candidate
by Benjamin Peterson March 7, 2010

March 7, 2010

On behalf of the Python development team, I'm pleased to announce a release candidate for the second bugfix release of the Python 3.1 series, Python 3.1.2. This bug fix release fixes numerous issues found in 3.1.1. This release candidate has been released to solicit testing and feedback over an possible regressions from 3.1.1. Please consider testing it with your library or application and reporting an bugs you encounter. This will help make the final 3.1.2 release, planned in 2 weeks time, all the more stable. The Python 3.1 version series focuses on the stabilization and optimization of the features and changes that Python 3.0 introduced. For example, the new I/O system has been rewritten in C for speed. File system APIs that use unicode strings now handle paths with undecodable bytes in them. Other features include an ordered dictionary implementation, a condensed syntax for nested with statements, and support for ttk Tile in Tkinter. For a more extensive list of changes in 3.1, see http://doc.python.org/3.1/whatsnew/3.1.html or Misc/NEWS in the Python distribution. To download Python 3.1.2rc1 visit: http://www.python.org/download/releases/3.1.2/ A list of changes in 3.1.2rc1 can be found here: http://svn.python.org/projects/python/tags/r312rc1/Misc/NEWS The 3.1 documentation can be found at: http://docs.python.org/3.1 Bugs can always be reported to: http://bugs.python.org Enjoy! -- Benjamin Peterson Release Manager benjamin at python.org (on behalf of the entire python-dev team and 3.1.2's contributors)

5 4