Mailman 3 February 2010 - Python-Dev

What to intern (e.g. func_code.co_filename)?
by Jake McGuire 13 Feb '10

13 Feb '10

Has anyone come up with rules of thumb for what to intern and what the performance implications of interning are? I'm working on profiling App Engine again, and since they don't allow marshall I have to modify pstats to save the profile via pickle. While trying to get profiles under 1MB, I noticed that each function has its own copy of the filename in which it is defined, and sometimes these strings can be rather long. Creating a code object already interns a bunch of stuff; argument names, variable names, etc. Interning the filename will add some CPU overhead during function creation, should save a decent amount of memory, and ought to have minimal overall performance impact. I have a local patch, but wanted to see if anyone had ideas or experience weighing these tradeoffs. -jake

3 2

Summary of Python tracker Issues
by Python tracker 12 Feb '10

12 Feb '10

ACTIVITY SUMMARY (02/05/10 - 02/12/10) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2602 open (+33) / 17137 closed (+25) / 19739 total (+58) Open issues with patches: 1069 Average duration of open issues: 712 days. Median duration of open issues: 466 days. Open Issues Breakdown open 2567 (+33) pending 34 ( +0) Issues Created Or Reopened (59) _______________________________ support "with self.assertRaises(SomeException) as exc:" syntax 02/07/10 CLOSED http://bugs.python.org/issue7859 reopened flox 32-bit Python on 64-bit Windows reports incorrect architecture 02/05/10 http://bugs.python.org/issue7860 created brian.curtin patch, patch, needs review 2to3: "import foo" -> "from . import foo" 02/05/10 CLOSED http://bugs.python.org/issue7861 created tomspur fileio.c: ValueError vs. IOError with impossible operations 02/05/10 http://bugs.python.org/issue7862 created skrah platform module doesn't detect Windows 7 02/05/10 http://bugs.python.org/issue7863 created adal patch, needs review Deprecation markers in unittest docs are unclear 02/06/10 CLOSED http://bugs.python.org/issue7864 created Justin.Lebar io close() swallowing exceptions 02/06/10 http://bugs.python.org/issue7865 created pakal it looks like a typo in unittest 02/06/10 CLOSED http://bugs.python.org/issue7866 created artech Proposed FAQ entry on pass-by-? semantics and the meaning of 'va 02/06/10 http://bugs.python.org/issue7867 created r.david.murray add a loggerClass attribute to Manager 02/06/10 CLOSED http://bugs.python.org/issue7868 created georg.brandl patch, patch, easy traceback from logging is unusable. 02/07/10 CLOSED http://bugs.python.org/issue7869 created naoki patch Duplicate test methods in test_memoryio 02/07/10 CLOSED http://bugs.python.org/issue7870 created georg.brandl Duplicate test method in test_heapq 02/07/10 http://bugs.python.org/issue7871 created georg.brandl Incorrect error raised on importing invalid module via unittest 02/07/10 CLOSED http://bugs.python.org/issue7872 created Daniel.Waterworth Remove precision restriction for integer formatting. 02/07/10 http://bugs.python.org/issue7873 created mark.dickinson easy logging.basicConfig should raise warning/exception on second cal 02/07/10 CLOSED http://bugs.python.org/issue7874 created tocomo test_multiprocessing / importlib failure 02/07/10 http://bugs.python.org/issue7875 created pitrou buildbot unittest docs use deprecated method in code example 02/07/10 CLOSED http://bugs.python.org/issue7876 created Bernt.Røskar.Brenna Iterators over _winreg EnumKey and EnumValue results 02/07/10 http://bugs.python.org/issue7877 created brian.curtin patch, patch, needs review regrtest should check for changes in import machinery 02/07/10 http://bugs.python.org/issue7878 created brett.cannon easy Too narrow platform check in test_datetime 02/07/10 http://bugs.python.org/issue7879 created akrpic77 patch sysconfig does not like symlinks 02/07/10 http://bugs.python.org/issue7880 created flox patch Hardcoded path, unsafe tempfile in test_logging 02/08/10 CLOSED http://bugs.python.org/issue7881 created nascheme Compiling on MOX 10.6 "Snow Leopard" --#warning Building for Int 02/08/10 CLOSED http://bugs.python.org/issue7882 created global667 CallTips.py _find_constructor does not work 02/08/10 http://bugs.python.org/issue7883 created Bernt.Røskar.Brenna IDLE 3.1.1 crashes with UnicodeDecodeError when I press Ctrl-Spa 02/08/10 CLOSED http://bugs.python.org/issue7884 created Bernt.Røskar.Brenna test_distutils fails if Python built in separate directory 02/08/10 http://bugs.python.org/issue7885 created nascheme reverse on an empty list returns None 02/08/10 CLOSED http://bugs.python.org/issue7886 created tormen errno 107 socket.recv issure 02/08/10 CLOSED http://bugs.python.org/issue7887 created twistedphrame turtle "settiltangle" should be marked deprecated, not "tiltangl 02/09/10 http://bugs.python.org/issue7888 created mnewman random produces different output on different architectures 02/09/10 http://bugs.python.org/issue7889 created terrence equal unicode/str objects can have unequal hash 02/09/10 CLOSED http://bugs.python.org/issue7890 created ldeller add links to SVN for documentation developers 02/09/10 http://bugs.python.org/issue7891 created techtonik refactor "test_dict.py" using new assertRaises context manager. 02/09/10 http://bugs.python.org/issue7892 created flox patch unittest: have to subclass TextTestRunner to use alternative Tes 02/09/10 CLOSED http://bugs.python.org/issue7893 created michael.foord easy too aggressive dependency tracking in distutils 02/09/10 http://bugs.python.org/issue7894 created ronaldoussoren Mac 10.6 mac_ver() crashes with USING_FORK_WITHOUT_EXEC_IS _NOT_ 02/09/10 http://bugs.python.org/issue7895 created aahz IDLE.app crashes when attempting to open a .py file 02/09/10 http://bugs.python.org/issue7896 created phyreman Support parametrized tests in unittest 02/10/10 http://bugs.python.org/issue7897 created fperez rlcompleter add "real tab" when text is empty feature 02/10/10 http://bugs.python.org/issue7898 created lilaboc patch MemoryError While Executing Python Code 02/10/10 http://bugs.python.org/issue7899 created p_noblebose posix.getgroups() failure on Mac OS X 02/10/10 http://bugs.python.org/issue7900 created michael.foord Add Vista/7 symlink support 02/10/10 CLOSED http://bugs.python.org/issue7901 created ipatrol relative import broken 02/10/10 http://bugs.python.org/issue7902 created gangesmaster Configure script incorrect for reasonably recent OpenBSD 02/10/10 http://bugs.python.org/issue7903 created johns urlparse.urlsplit mishandles novel schemes 02/10/10 http://bugs.python.org/issue7904 created mbloore Shelf 'keyencoding' keyword argument is undocumented and does no 02/11/10 http://bugs.python.org/issue7905 created r.david.murray patch, patch, needs review float("INFI") returns inf on certain platforms 02/11/10 CLOSED http://bugs.python.org/issue7906 created csernazs winreg docs: CreateKeyEx should be CreateKey (minor) 02/11/10 CLOSED http://bugs.python.org/issue7907 created patraulea patch, needs review remove leftover macos9 support code 02/11/10 http://bugs.python.org/issue7908 created ronaldoussoren patch, needs review os.path.abspath(os.devnull) returns \\\\nul should be nul? 02/11/10 http://bugs.python.org/issue7909 created abakker immutability w/r to tuple.__add__ 02/11/10 CLOSED http://bugs.python.org/issue7910 created mattrussell unittest.TestCase.longMessage should default to True in Python 3 02/11/10 http://bugs.python.org/issue7911 created michael.foord easy Error in additon of decimal numbers 02/11/10 CLOSED http://bugs.python.org/issue7912 created James.Sparenberg Enhance Cmd support for docstrings and document it. 02/11/10 http://bugs.python.org/issue7913 created r.david.murray patch IntVar() - AttributeError: 'NoneType' object has no attribute 't 02/11/10 CLOSED http://bugs.python.org/issue7914 created Plazma A lists which list.sort seems to leave out of order. 02/12/10 CLOSED http://bugs.python.org/issue7915 created throwaway zipfile.ZipExtFile passes long to fileobj.read() 02/12/10 CLOSED http://bugs.python.org/issue7916 created anacrolix list of list created by * 02/12/10 CLOSED http://bugs.python.org/issue7917 created sledge76 Issues Now Closed (60) ______________________ Intermitent failure in test_multiprocessing.test_number_of_objec 543 days http://bugs.python.org/issue3562 flox allow unicode keyword args 2 days http://bugs.python.org/issue4978 benjamin.peterson patch, needs review A selection of spelling errors and typos throughout source 3 days http://bugs.python.org/issue5341 merwok patch [PATCH]Add FastDbfilenameShelf: shelf nerver sync cache even whe 335 days http://bugs.python.org/issue5483 r.david.murray patch memory leaks in py3k 317 days http://bugs.python.org/issue5596 flox patch Serious interpreter crash and/or arbitrary memory leak using .re 309 days http://bugs.python.org/issue5677 pitrou patch Shelve module writeback parameter does not act as advertised 302 days http://bugs.python.org/issue5754 r.david.murray patch, easy ZipFile.writestr "compression_type" argument 271 days http://bugs.python.org/issue6003 ronaldoussoren patch ElementTree (py3k) doesn't properly encode characters that can't 247 days http://bugs.python.org/issue6233 pitrou patch rare assertion failure in test_multiprocessing 224 days http://bugs.python.org/issue6366 flox OS X: python3 from python-3.1.dmg crashes at startup 224 days http://bugs.python.org/issue6393 srid patch, needs review multiprocessing logging support test 77 days http://bugs.python.org/issue6615 vinay.sajip patch test test_multiprocessing failed 172 days http://bugs.python.org/issue6747 flox enable compilation of readline module on Mac OS X 10.5 and 10.6 7 days http://bugs.python.org/issue6877 minge patch, 26backport, needs review Update version{added,changed} entries in py3k unittest docs 127 days http://bugs.python.org/issue7030 ezio.melotti Make assertMultilineEqual default for unicode string comparison 130 days http://bugs.python.org/issue7032 michael.foord low performance of zipfile readline() 105 days http://bugs.python.org/issue7216 amaury.forgeotdarc OverflowError: signed integer is greater than maximum on mips64 91 days http://bugs.python.org/issue7296 mark.dickinson OS X 2.6.4 installer fails on 10.3 with two corrupted file names 64 days http://bugs.python.org/issue7437 ronaldoussoren python -m unittest path_to_suite_function errors 59 days http://bugs.python.org/issue7501 michael.foord unittest.TestCase.shortDescription isn't short anymore 44 days http://bugs.python.org/issue7588 michael.foord OS X pythonw.c compile error with 10.4 or earlier deployment tar 30 days http://bugs.python.org/issue7658 ronaldoussoren patch configure GCC version detection fix for Darwin 22 days http://bugs.python.org/issue7714 ronaldoussoren patch, needs review Allow use of GNU arch on Darwin 26 days http://bugs.python.org/issue7715 ronaldoussoren patch, needs review test_timeout should use "find_unused_port" helper 19 days http://bugs.python.org/issue7728 r.david.murray patch, easy, buildbot unittest returning standard_tests from load_tests in module fail 8 days http://bugs.python.org/issue7799 michael.foord easy test_macostools fails on OS X 10.6: no attribute 'FSSpec' 9 days http://bugs.python.org/issue7807 ronaldoussoren Call to gestalt('sysu') on OSX can lead to freeze in wxPython ap 8 days http://bugs.python.org/issue7812 ronaldoussoren Minor bug in 2.6.4 related to cleanup at end of program 9 days http://bugs.python.org/issue7835 r.david.murray patch, easy python-dev archives are not updated 5 days http://bugs.python.org/issue7843 isandler Remove 'python -U' or document it 2 days http://bugs.python.org/issue7847 barry patch, needs review copy.copy corrupts objects that return false value from __getsta 4 days http://bugs.python.org/issue7848 georg.brandl on __exit__(), exc_value does not contain the exception. 2 days http://bugs.python.org/issue7853 benjamin.peterson patch test_logging fails 3 days http://bugs.python.org/issue7857 vinay.sajip buildbot support "with self.assertRaises(SomeException) as exc:" syntax 0 days http://bugs.python.org/issue7859 flox 2to3: "import foo" -> "from . import foo" 3 days http://bugs.python.org/issue7861 tomspur Deprecation markers in unittest docs are unclear 0 days http://bugs.python.org/issue7864 georg.brandl it looks like a typo in unittest 0 days http://bugs.python.org/issue7866 merwok add a loggerClass attribute to Manager 0 days http://bugs.python.org/issue7868 georg.brandl patch, patch, easy traceback from logging is unusable. 0 days http://bugs.python.org/issue7869 vinay.sajip patch Duplicate test methods in test_memoryio 0 days http://bugs.python.org/issue7870 pitrou Incorrect error raised on importing invalid module via unittest 0 days http://bugs.python.org/issue7872 michael.foord logging.basicConfig should raise warning/exception on second cal 0 days http://bugs.python.org/issue7874 vinay.sajip unittest docs use deprecated method in code example 1 days http://bugs.python.org/issue7876 ezio.melotti Hardcoded path, unsafe tempfile in test_logging 0 days http://bugs.python.org/issue7881 vinay.sajip Compiling on MOX 10.6 "Snow Leopard" --#warning Building for Int 1 days http://bugs.python.org/issue7882 ronaldoussoren IDLE 3.1.1 crashes with UnicodeDecodeError when I press Ctrl-Spa 0 days http://bugs.python.org/issue7884 ezio.melotti reverse on an empty list returns None 0 days http://bugs.python.org/issue7886 tormen errno 107 socket.recv issure 1 days http://bugs.python.org/issue7887 twistedphrame equal unicode/str objects can have unequal hash 0 days http://bugs.python.org/issue7890 lemburg unittest: have to subclass TextTestRunner to use alternative Tes 1 days http://bugs.python.org/issue7893 michael.foord easy Add Vista/7 symlink support 0 days http://bugs.python.org/issue7901 tim.golden float("INFI") returns inf on certain platforms 1 days http://bugs.python.org/issue7906 r.david.murray winreg docs: CreateKeyEx should be CreateKey (minor) 0 days http://bugs.python.org/issue7907 ezio.melotti patch, needs review immutability w/r to tuple.__add__ 0 days http://bugs.python.org/issue7910 rhettinger Error in additon of decimal numbers 0 days http://bugs.python.org/issue7912 r.david.murray IntVar() - AttributeError: 'NoneType' object has no attribute 't 0 days http://bugs.python.org/issue7914 brian.curtin A lists which list.sort seems to leave out of order. 0 days http://bugs.python.org/issue7915 throwaway zipfile.ZipExtFile passes long to fileobj.read() 0 days http://bugs.python.org/issue7916 amaury.forgeotdarc list of list created by * 0 days http://bugs.python.org/issue7917 flox Top Issues Most Discussed (10) ______________________________ 17 Add a context manager to change cwd in test.test_support and ru 27 days open http://bugs.python.org/issue7712 17 enable compilation of readline module on Mac OS X 10.5 and 10.6 7 days closed http://bugs.python.org/issue6877 10 random produces different output on different architectures 4 days open http://bugs.python.org/issue7889 9 Support parametrized tests in unittest 3 days open http://bugs.python.org/issue7897 9 support "with self.assertRaises(SomeException) as exc:" syntax 0 days closed http://bugs.python.org/issue7859 8 test_multiprocessing / importlib failure 5 days open http://bugs.python.org/issue7875 8 Add os.link() and os.symlink() and os.path.islink() support for 1215 days open http://bugs.python.org/issue1578269 7 float("INFI") returns inf on certain platforms 1 days closed http://bugs.python.org/issue7906 7 Proposed FAQ entry on pass-by-? semantics and the meaning of 'v 6 days open http://bugs.python.org/issue7867 7 platform module doesn't detect Windows 7 7 days open http://bugs.python.org/issue7863

1 0

unittest: shortDescription, _TextTestResult and other issues
by Michael Foord 11 Feb '10

11 Feb '10

Hello all, I've been looking at outstanding unittest issues as part of my preparation for my PyCon talk. There are a couple of changes (minor) I'd like to make that I thought I ought to run past Python-Dev first. If I don't get any responses then I'll just do it, so you have been warned. :-) The great google merge into unittest happened at PyCon last year [1]. This included a change to TestCase.shortDescription() so that it would *always* include the test name, whereas previously it would return the test docstring or None. The problem this change solved was that tests with a docstring would not have their name (test class and method name) reported during the test run. Unfortunately the change broke part of twisted test running. Reported as issue 7588: http://bugs.python.org/issue7588 It seems to me that the same effect (always reporting test name) can be achieved in _TextTestResult.getDescription(). I propose to revert the change to TestCase.shortDescription() (which has both a horrible name and a horrible implementation and should probably be renamed getDocstring so that what it does is obvious but never mind) and put the change into _TextTestResult. It annoys me that _TextTestResult is private, as you will almost certainly want to use it or subclass it when implementing custom test systems. I am going to rename it TextTestResult, alias the old name and document the old name as being deprecated. Another issue that I would like to address, but there are various possible approaches, is issue 7559: http://bugs.python.org/issue7559 Currently loadTestsFromName catches ImportError and rethrows as AttributeError. This is horrible (it obscures the original error) but there are backwards compatibility issues with fixing it. There are three possible approaches: 1) Leave it (the default) 2) Only throw an AttributeError if the import fails due to the name being invalid (the module not existing) otherwise allow the error through. (A minor but less serious change in behavior). 3) A new method that turns failures into pseudo-tests that fail with the original error when run. Possibly deprecating loadTestsFromName I favour option 3, but can't think of a good replacement name. :-) Comments welcomed. Despite deprecating (in the documentation - no actual deprecations warnings I believe) a lot of the duplicate ways of doing things (assert* favoured over fail* and assertEqual over assertEquals) we didn't include deprecating assert_ in favour of assertTrue. I would like to add that to the documentation. After 3.2 is out I would like to clean up the documentation, removing mention of the deprecated methods from the *main* documentation into a separate 'deprecated methods' section. They currently make the documentation very untidy. The unittest page should probably be split into several pages anyway and needs improving. Other outstanding minor issues: Allow dotted names for test discovery http://bugs.python.org/issue7780 - I intend to implement this as described in the last comment A 'check_order' optional argument (defaulting to True) for assertSequenceEqual http://bugs.python.org/issue7832 - needs patch The breaking of __unittest caused by splitting unittesst into a package needs fixing. The fix needs to work when Python is run without frames support (IronPython). http://bugs.python.org/issue7815 - needs patch Allow a __unittest (or similar) decorator for user implemented assert functions http://bugs.python.org/issue1705520 - needs patch Allow modules to define test_suite callable. http://bugs.python.org/issue7501 - I propose to close as rejected. Use load_tests instead. Display time taken by individual tests when in verbose mode. http://bugs.python.org/issue4080 - anyone any opinions? Allow automatic formatting of arguments in assert* failure messages. http://bugs.python.org/issue6966 - I propose to close as rejected removeTest() method on TestSuite http://bugs.python.org/issue1778410 - anyone any opinions? expect methods (delayed fail) http://bugs.python.org/issue3615 - any opinions? Personally I think that the TestCase API is big enough already All the best, Michael Foord [1] Mostly in revision 7-837. http://svn.python.org/view?view=rev&revision=70837 -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

6 13

Re: [Python-Dev] Python 2.6.5
by anatoly techtonik 11 Feb '10

11 Feb '10

On Tue, Feb 2, 2010 at 8:08 PM, Barry Warsaw <barry(a)python.org> wrote: > I'm thinking about doing a Python 2.6.5 release soon. I've added the > following dates to the Python release schedule Google calendar: > > 2009-03-01 Python 2.6.5 rc 1 > 2009-03-15 Python 2.6.5 final > > This allows us to spend some time on 2.6.5 at Pycon if we want. Please let me > know if you have any concerns about those dates. I've noticed a couple of issues that 100% crash Python 2.6.4 like this one - http://bugs.python.org/issue6608 Is it ok to release new versions that are known to crash? -- anatoly t.

10 30

Re: [Python-Dev] Executing zipfiles and directories (was Re: PyCon Keynote)
by Glenn Linderman 11 Feb '10

11 Feb '10

On approximately 1/27/2010 1:08 AM, came the following characters from the keyboard of Glenn Linderman: > Without reference to distutils, it seems the pieces are: > > 1) a way to decide what to include in the package > 2) code that knows where to put what is included, on one or more > platforms > 3) the process to create the ZIP file that includes 1 & 2, and call it > an appropriate name.py > > 3 looks easy, once 1 & 2 are figured out. distutils might provide the > foundation for 1. 2 sounds like something a distutils application > might create. I'm not sure that distutils is in the business of > building installer programs, my understanding is that it is in the > business of providing a standard way of recording and interpreting > bills of materials and maybe dependencies, but that is based only on > reading discussions here, not reading documentation. I haven't had a > chance to read all the module documentation since coming to python. 3 was rather easy, and has come in handy for me, as it turned out: see http://code.activestate.com/recipes/577042/ -- Glenn -- http://nevcal.com/ =========================== A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking

1 0

Re: [Python-Dev] PEP 3146: Merge Unladen Swallow into CPython
by Collin Winter 10 Feb '10

10 Feb '10

To follow up on some of the open issues: On Wed, Jan 20, 2010 at 2:27 PM, Collin Winter <collinwinter(a)google.com> wrote: [snip] > Open Issues > =========== > > - *Code review policy for the ``py3k-jit`` branch.* How does the CPython > community want us to procede with respect to checkins on the ``py3k-jit`` > branch? Pre-commit reviews? Post-commit reviews? > > Unladen Swallow has enforced pre-commit reviews in our trunk, but we realize > this may lead to long review/checkin cycles in a purely-volunteer > organization. We would like a non-Google-affiliated member of the CPython > development team to review our work for correctness and compatibility, but we > realize this may not be possible for every commit. The feedback we've gotten so far is that at most, only larger, more critical commits should be sent for review, while most commits can just go into the branch. Is that broadly agreeable to python-dev? > - *How to link LLVM.* Should we change LLVM to better support shared linking, > and then use shared linking to link the parts of it we need into CPython? The consensus has been that we should link shared against LLVM. Jeffrey Yasskin is now working on this in upstream LLVM. We are tracking this at http://code.google.com/p/unladen-swallow/issues/detail?id=130 and http://llvm.org/PR3201. > - *Prioritization of remaining issues.* We would like input from the CPython > development team on how to prioritize the remaining issues in the Unladen > Swallow codebase. Some issues like memory usage are obviously critical before > merger with ``py3k``, but others may fall into a "nice to have" category that > could be kept for resolution into a future CPython 3.x release. The big-ticket items here are what we expected: reducing memory usage and startup time. We also need to improve profiling options, both for oProfile and cProfile. > - *Create a C++ style guide.* Should PEP 7 be extended to include C++, or > should a separate C++ style PEP be created? Unladen Swallow maintains its own > style guide [#us-styleguide]_, which may serve as a starting point; the > Unladen Swallow style guide is based on both LLVM's [#llvm-styleguide]_ and > Google's [#google-styleguide]_ C++ style guides. Any thoughts on a CPython C++ style guide? My personal preference would be to extend PEP 7 to cover C++ by taking elements from http://code.google.com/p/unladen-swallow/wiki/StyleGuide and the LLVM and Google style guides (which is how we've been developing Unladen Swallow). If that's broadly agreeable, Jeffrey and I will work on a patch to PEP 7. Thanks, Collin Winter

2 1

PyCon is coming! Tomorrow, Feb. 10th is the last day for pre-conference rates
by VanL 09 Feb '10

09 Feb '10

PyCon is coming! Tomorrow (February 10th) is the last day for pre-conference rates. You can register for PyCon online at: <https://us.pycon.org/2010/register/> Register while it is still Feb. 10th somewhere in the world and rest easy in the knowledge that within 10 days you will enjoying the company of some of the finest Python hackers in the world. As an additional bonus, PyCon this year will be in Atlanta, making it an ideal location for those looking for a way to escape the late winter blizzards in the northeastern United States, or the dreary fog of the Bay area. See you at PyCon 2010!

1 0

PEP 345 and PEP 386
by Guido van Rossum 09 Feb '10

09 Feb '10

All, I've reviewed PEP 345 and PEP 386 and am satisfied that after some small improvements they will be accepted. Most of the discussion has already taken place. I have one comment on PEP 345: Why is author-email mandatory? I'm sure there are plenty of cases where either the author doesn't want their email address published, or their last know email address is no longer valid. (Tarek responded off-line that it isn't all that mandatory; I propose to say so in the PEP.) I am also looking at PEP 376 but I expect that Tarek will start another round of discussion on it. Hopefully all three PEPs will be accepted in time for inclusion in Python 2.7. -- --Guido van Rossum (python.org/~guido)

2 3

Request for review of issue 4037
by Steven D'Aprano 09 Feb '10

09 Feb '10

Hello, I have submitted a patch and a test script for issue 4037 on the bug tracker, "doctest.py should include method descriptors when looking inside a class __dict__" http://bugs.python.org/issue4037 I would be grateful if somebody could review it please, and if suitable, commit it. Thank you. -- Steven D'Aprano

1 0

Re: [Python-Dev] Forking and Multithreading - enemy brothers
by Ben Walker 09 Feb '10

09 Feb '10

Pascal Chambon writes: > I don't really get it there... it seems to me that multiprocessing only > requires picklability for the objects it needs to transfer, i.e those > given as arguments to the called function, and thsoe put into > multiprocessing queues/pipes. Global program data needn't be picklable - > on windows it gets wholly recreated by the child process, from python > bytecode. > > So if you're having pickle errors, it must be because the > "object_from_module_xyz" itself is *not* picklable, maybe because it > contains references to unpicklable objets. In such case, properly > implementing pickle magic methods inside the object should do it, > shouldn't it ? I'm also a long time lurker (and in financial software, coincidentally). Pascal is correct here. We use a number of C++ libraries wrapped via Boost.Python to do various calculations. The typical function calls return wrapped C++ types. Boost.Python types are not, unfortunately, pickleable. For a number of technical reasons, and also unfortunately, we often have to load these libraries in their own process, but we want to hide this from our users. We accomplish this by pickling the instance data, but importing the types fresh when we unpickle, all implemented in the magic pickle methods. We would lose any information that was dynamically added to the type in the remote process, but we simply don't do that. We very often have many unpickleable objects imported somewhere when we spin off our processes using the multiprocess library, and this does not cause any problems. Jesse Noller <jnoller <at> gmail.com> writes: > We already have an implementation that spawns a > subprocess and then pushes the required state to the child. The > fundamental need for things to be pickleable *all the time* kinda > makes it annoying to work with. This requirement puts a fairly large additional strain on working with unwieldy, wrapped C++ libraries in a multiprocessing environment. I'm not very knowledgeable on the internals of the system, but would it be possible to have some kind of fallback system whereby if an object fails to pickle we instead send information about how to import it? This has all kinds of limitations - it only works for importable things (i.e. not instances), it can potentially lose information dynamically added to the object, etc., but I thought I would throw the idea out there so someone knowledgeable can decide if it has any merit. Ben

3 2