Mailman 3 October 2009 - Python-Dev

Re: [Python-Dev] tokenize string literal problem
by C or L Smith 24 Oct '09

24 Oct '09

C or L Smith wrote: > PROBLEM > I need to find code snippets which are located in docstrings. > Docstrings, being string literals should be able to be parsed out > with tokenize. But tokenize is giving the wrong results (or I am > doing something wrong) for this (pathological) case: > > foo.py: > +---- > def bar(): > """ > A quoted triple quote is not a closing > of this docstring: > >>> print '"""' > """ > """ # <-- this is the closing quote > pass > +---- > I now see that I've created a code snippet that is invalid. Myopia. The thing that pythonWin was doing correctly was displaying my sample STRING not code. I had delimited the code with triple-single-quotes so it showed up correctly. In fact, if entered as code it would show the need to delimit the docstring contents with ''' rather than """. Sorry! /c

1 0

tokenize string literal problem
by C or L Smith 24 Oct '09

24 Oct '09

BACKGROUND I'm trying to modify the doctest DocTestParser so it will parse docstring code snippets out of a *py file. (Although doctest can parse these with another method out of *pyc, it is missing certain decorated functions and we would also like to insist of import of needed modules rather and that method automatically loads everything from the module containing the code.) PROBLEM I need to find code snippets which are located in docstrings. Docstrings, being string literals should be able to be parsed out with tokenize. But tokenize is giving the wrong results (or I am doing something wrong) for this (pathological) case: foo.py: +---- def bar(): """ A quoted triple quote is not a closing of this docstring: >>> print '"""' """ """ # <-- this is the closing quote pass +---- Here is how I tokenize the file: ### import re, tokenize DOCSTRING_START_RE = re.compile('\s+[ru]*("""|' + "''')") o=open('foo.py','r') for ti in tokenize.generate_tokens(o.next): typ = ti[0] text = ti[-1] if typ == tokenize.STRING: if DOCSTRING_START_RE.match(text): print "DOCSTRING:",repr(text) o.close() ### which outputs: DOCSTRING: ' """\n A quoted triple quote is not a closing\n of this docstring:\n >>> print \'"""\'\n' DOCSTRING: ' """\n """ # <-- this is the closing quote\n' There should be only one string tokenized, I believe. The PythonWin editor parses (and colorizes) this correctly, but tokenize (or I) are making an error. Thanks for any help, Chris

1 0

Re: [Python-Dev] [Python-ideas] Remove GIL with CAS instructions?
by Kristján Valur Jónsson 23 Oct '09

23 Oct '09

> -----Original Message----- > From: python-ideas-bounces+kristjan=ccpgames.com(a)python.org > [mailto:python-ideas-bounces+kristjan=ccpgames.com@python.org] On > Behalf Of Sturla Molden > Sent: 20. október 2009 22:13 > To: python-ideas(a)python.org > Subject: Re: [Python-ideas] Remove GIL with CAS instructions? > > > - The GIL has consequences on multicore CPUs that are overlooked: > thread > switches are usually missed at check intervals. This could be fixed > without removing the GIL: For example, there could be a wait-queue for > the GIL; a thread that request the GIL puts itself in the back. > This depends entirely on the platform and primitives used to implement the GIL. I'm interested in windows. There, I found this article: http://fonp.blogspot.com/2007/10/fairness-in-win32-lock-objects.html So, you may be on to something. Perhaps a simple C test is in order then? I did that. I found, on my dual-core vista machine, that running "release", that both Mutexes and CriticalSections behaved as you describe, with no "fairness". Using a "semaphore" seems to retain fairness, however. "fairness" was retained in debug builds too, strangely enough. Now, Python uses none of these. On windows, it uses an "Event" object coupled with an atomically updated counter. This also behaves fairly. The test application is attached. I think that you ought to sustantiate your claims better, maybe with a specific platform and using some test like the above. On the other hand, it shows that we must be careful what we use. There has been some talk of using CriticalSections for the GIL on windows. This test ought to show the danger of that. The GIL is different than a regular lock. It is a reverse-lock, really, and therefore may need to be implemented in its own special way, if we want very fast mutexes for the rest of the system (cc to python-dev) Cheers, Kristján

14 43

Bug 7183 and Python 2.6.4
by Barry Warsaw 23 Oct '09

23 Oct '09

I'd like to get a second opinion on bug 7183: http://bugs.python.org/issue7183 The Boost folks have reported this as a regression in 2.6.3, making it a candidate for Python 2.6.4. IIUC, the latest version of Boost fixes the problem in their code, but if it really is a regression it could affect other projects and should probably be fixed. Robert Collins from the Bazaar team pinged me on IRC and originally thought that they'd been bitten by this problem too, though he looked around and determined it probably did /not/ affect them after all. So we have only anecdotal evidence of a problem, and no reproducible test case. If the Python 2.6.4 release is to be held up for this issue, we need to know today or tomorrow. Come the weekend, I'm going ahead with the tag and release. Holding up the release will mean another release candidate, and a wait of another week for the final. So does anybody else think bug 7183 should be a release blocker for 2.6.4 final, or is even a legitimate but that we need to fix? Thanks, -Barry

17 28

Volunteer needed to organize summits
by A.M. Kuchling 23 Oct '09

23 Oct '09

I'd like to turn over the organization of the VM and Python Language Summits at PyCon 2010 to someone else, one or two people. (The same person doesn't need to organize both of them.) Why: in November PyCon will be three months away, so the guest list needs to be finalized and the invitations need to be sent. Yet I can't pull together the motivation to work on them; I contemplate the task for two minutes and then go do something else. It's obviously better if the summit tasks are being actively worked on instead of just drifting, so I want to give it up now. What's required: chiefly it's just a matter of sending and replying to e-mail. Draw up a guest list (I can provide last year's lists); think of new people & projects to be added, or e-mail someone to ask for suggestions; send out invitations and requests for agenda items; collect the responses so we know how many people are coming. You can also help moderate the summits on the day of the events, but if that's not feasible someone else could do it, or the groups could manage themselves. (Also sent to pycon-organizers, psf-members.) --amk

3 2

Summary of Python tracker Issues
by Python tracker 23 Oct '09

23 Oct '09

ACTIVITY SUMMARY (10/16/09 - 10/23/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2474 open (+27) / 16541 closed (+16) / 19015 total (+43) Open issues with patches: 987 Average duration of open issues: 673 days. Median duration of open issues: 428 days. Open Issues Breakdown open 2436 (+27) pending 37 ( +0) Issues Created Or Reopened (43) _______________________________ regrtest -j sometimes fails if output gets written to stderr by 10/16/09 CLOSED http://bugs.python.org/issue7151 created r.david.murray patch, patch urllib2.build_opener() skips ProxyHandler 10/16/09 http://bugs.python.org/issue7152 created barry add "start" arg to max and min functions 10/16/09 CLOSED http://bugs.python.org/issue7153 created phr urllib.request system proxy configuration lookup broken for OS X 10/16/09 http://bugs.python.org/issue7154 created ned.deily urllib2 and python 3 urllib do not document default use of syste 10/16/09 CLOSED http://bugs.python.org/issue7155 created ned.deily patch curses can't find _curses 10/17/09 CLOSED http://bugs.python.org/issue7156 created Archon Fix Download Current Documentation link 10/17/09 http://bugs.python.org/issue7157 created ryles os.path.basename/split fails 10/17/09 CLOSED http://bugs.python.org/issue7158 created kuiper Urllib2 authentication memory. 10/17/09 http://bugs.python.org/issue7159 created bradobro Crash when returning a 64-bit char pointer in Python 2.6.3 ctype 10/17/09 CLOSED http://bugs.python.org/issue7160 created creachadair raise of SyntaxError in codeop was ported incorrectly to Py3 10/17/09 CLOSED http://bugs.python.org/issue7161 created Trundle patch 2to3 does not convert __builtins__.file 10/17/09 http://bugs.python.org/issue7162 created joe.amenta IDLE suppresses sys.stdout.write() return value 10/17/09 http://bugs.python.org/issue7163 created tjreedy pickle test failure after test_imp/test_import (_make_stat_resul 10/18/09 CLOSED http://bugs.python.org/issue7164 created r.david.murray xmlrpc.server assumes sys.stdout will have a buffer attribute 10/18/09 http://bugs.python.org/issue7165 created ncoghlan patch IDLE (python 3.1.1) syntax coloring for b'bytestring' and u'unic 10/18/09 http://bugs.python.org/issue7166 created lieryan easy Smarter FTP passive mode 10/18/09 http://bugs.python.org/issue7167 created pitrou patch Document PyFloat_AsString and PyFloat_AsReprString as deprecated 10/19/09 CLOSED http://bugs.python.org/issue7168 created eric.smith patch zipfile leaves a file handle open if file is zero size 10/19/09 http://bugs.python.org/issue7169 created skelker subclasses of (some) built-in types are not weakref-able 10/19/09 CLOSED http://bugs.python.org/issue7170 created stutzbach Add inet_ntop and inet_pton support for Windows 10/19/09 http://bugs.python.org/issue7171 created jaraco BaseHTTPServer.BaseHTTPRequestHandler.responses[405] has a small 10/19/09 http://bugs.python.org/issue7172 created dalke Cython compiler run crashes CPython 3.1.1 and 3.2 10/20/09 http://bugs.python.org/issue7173 created scoder patch modeule: queue class: PriorityQueue 10/20/09 CLOSED http://bugs.python.org/issue7174 created asuiu unify pydistutils.cfg and distutils.cfg and use .local 10/20/09 http://bugs.python.org/issue7175 created tarek sum() doesn't work for lists. 10/20/09 CLOSED http://bugs.python.org/issue7176 created oggust Unclear warning for subprocess.call 10/20/09 http://bugs.python.org/issue7177 created azumanga open function's buffering parameter is not completely doc'ed 10/20/09 CLOSED http://bugs.python.org/issue7178 created kuhnsjohn Unportable test(1) construct 10/21/09 http://bugs.python.org/issue7179 created wiz patch "pydoc -k" can generate AttributeError on Mac OS X 10/21/09 http://bugs.python.org/issue7180 created kmgrant No logging when two loggers with same qualname 10/21/09 CLOSED http://bugs.python.org/issue7181 created cristiroma For non-debug builds, the cygwinccompiler.py should define NDEBU 10/21/09 http://bugs.python.org/issue7182 created stutzbach did 2.6.3 regress for some uses of the __doc__ property? 10/21/09 http://bugs.python.org/issue7183 created zooko build failures on Snow Leopard 10/22/09 http://bugs.python.org/issue7184 created richard csv reader utf-8 BOM error 10/22/09 http://bugs.python.org/issue7185 created W00D00 Document specialness of __doc__, and possibly other "special" at 10/22/09 http://bugs.python.org/issue7186 created ssteiner importlib/_bootstrap.py write_bytecode raises an IOError if it c 10/22/09 http://bugs.python.org/issue7187 created dmalcolm patch optionxform documentation confusing 10/23/09 CLOSED http://bugs.python.org/issue7188 created loewis struct.calcsize returns strange size 10/23/09 CLOSED http://bugs.python.org/issue7189 created igor.mikushkin TCP/IP? 10/23/09 http://bugs.python.org/issue7190 created jlr2383 Odd behaviour with zlib.decompressobj optional parameter "wbits" 10/23/09 http://bugs.python.org/issue7191 created pythonhacker webbrowser.get("firefox") does not work on Mac with installed Fi 10/23/09 http://bugs.python.org/issue7192 created dalke patch Popen blocks program from another thread 10/23/09 http://bugs.python.org/issue7193 created dgriff1 Issues Now Closed (41) ______________________ fromfd() and dup() for _socket on WIndows 716 days http://bugs.python.org/issue1378 planders patch GC optimization: don't track simple tuples and dicts 303 days http://bugs.python.org/issue4688 esam patch optimize bytecode for conditional branches 299 days http://bugs.python.org/issue4715 esam patch Bugs in _ssl object read() when a buffer is specified 278 days http://bugs.python.org/issue4967 pitrou patch "in" expression falls back to __iter__ before __getitem__ 122 days http://bugs.python.org/issue6324 georg.brandl Printing the 'The Python Tutorial' 76 days http://bugs.python.org/issue6670 brimac uuid creates zombies 42 days http://bugs.python.org/issue6882 eric.smith patch Metaclass doc (Ref 3.3.3) errors 36 days http://bugs.python.org/issue6927 georg.brandl Getopt documentation ambiguity 29 days http://bugs.python.org/issue6977 georg.brandl codecs error handlers lack documentation 20 days http://bugs.python.org/issue7035 georg.brandl Doc/reference/datamodel: Slots description needs update 20 days http://bugs.python.org/issue7036 georg.brandl patch urllib.urlopen crashes when used on Mac OS 10.6.1 through a prox 15 days http://bugs.python.org/issue7044 ronaldoussoren No docs for module 'IN' 17 days http://bugs.python.org/issue7062 georg.brandl Python 2.6.3 / setuptools 0.6c9: extension module builds fail wi 6 days http://bugs.python.org/issue7064 ned.deily Suggest better documentation of 2 versus 3 on 3.1.1 site. 17 days http://bugs.python.org/issue7067 srl locale.strxfrm raises MemoryError 12 days http://bugs.python.org/issue7080 pitrou patch New (in 2.6) functions in signal module are not documented as un 13 days http://bugs.python.org/issue7088 georg.brandl Problems building pyhton from source on Snow Leopard (Mac OS X 7 days http://bugs.python.org/issue7102 ronaldoussoren regrtest -j fails when tests write to stderr 4 days http://bugs.python.org/issue7127 r.david.murray easy test_ssl failure 5 days http://bugs.python.org/issue7133 pitrou patch Socket documentation not updated 7 days http://bugs.python.org/issue7137 georg.brandl Remove WITHOUT_COMPLEX from 3.x trunk 3 days http://bugs.python.org/issue7147 skip.montanaro patch, patch, easy, needs review 2.6.4rc1 regression: test_urllib2 fails on OS X with UnboundLoca 3 days http://bugs.python.org/issue7149 ned.deily regrtest -j sometimes fails if output gets written to stderr by 3 days http://bugs.python.org/issue7151 r.david.murray patch, patch add "start" arg to max and min functions 2 days http://bugs.python.org/issue7153 rhettinger urllib2 and python 3 urllib do not document default use of syste 1 days http://bugs.python.org/issue7155 orsenthil patch curses can't find _curses 6 days http://bugs.python.org/issue7156 georg.brandl os.path.basename/split fails 0 days http://bugs.python.org/issue7158 ezio.melotti Crash when returning a 64-bit char pointer in Python 2.6.3 ctype 2 days http://bugs.python.org/issue7160 creachadair raise of SyntaxError in codeop was ported incorrectly to Py3 0 days http://bugs.python.org/issue7161 benjamin.peterson patch pickle test failure after test_imp/test_import (_make_stat_resul 0 days http://bugs.python.org/issue7164 ncoghlan Document PyFloat_AsString and PyFloat_AsReprString as deprecated 1 days http://bugs.python.org/issue7168 eric.smith patch subclasses of (some) built-in types are not weakref-able 1 days http://bugs.python.org/issue7170 georg.brandl modeule: queue class: PriorityQueue 0 days http://bugs.python.org/issue7174 r.david.murray sum() doesn't work for lists. 0 days http://bugs.python.org/issue7176 rhettinger open function's buffering parameter is not completely doc'ed 0 days http://bugs.python.org/issue7178 benjamin.peterson No logging when two loggers with same qualname 0 days http://bugs.python.org/issue7181 vinay.sajip optionxform documentation confusing 0 days http://bugs.python.org/issue7188 georg.brandl struct.calcsize returns strange size 0 days http://bugs.python.org/issue7189 mark.dickinson os.path, %HOME% set: realpath contradicts expanduser on '~' 993 days http://bugs.python.org/issue1646838 ezio.melotti PyThreadState_SetAsyncExc and the main thread 788 days http://bugs.python.org/issue1779233 pitrou Top Issues Most Discussed (10) ______________________________ 10 add ftp-tls support to ftplib - RFC 4217 623 days open http://bugs.python.org/issue2054 9 did 2.6.3 regress for some uses of the __doc__ property? 2 days open http://bugs.python.org/issue7183 9 unify pydistutils.cfg and distutils.cfg and use .local 3 days open http://bugs.python.org/issue7175 9 Add gamma function, error functions and other C99 math.h functi 465 days open http://bugs.python.org/issue3366 6 add "start" arg to max and min functions 2 days closed http://bugs.python.org/issue7153 6 locale.strxfrm raises MemoryError 12 days closed http://bugs.python.org/issue7080 5 subclasses of (some) built-in types are not weakref-able 1 days closed http://bugs.python.org/issue7170 5 os.path.basename/split fails 0 days closed http://bugs.python.org/issue7158 5 2.6.4rc1 regression: test_urllib2 fails on OS X with UnboundLoc 3 days closed http://bugs.python.org/issue7149 5 deprecated conversion from string constant to char * 33 days open http://bugs.python.org/issue6952

1 0

PyErr_NewExceptionWithDoc
by Georg Brandl 22 Oct '09

22 Oct '09

Hi, issue #7033 proposes a new C API that creates a new exception class with a docstring. Since exception classes should be documented, and adding the docstring after creating is not a one-liner, I would say it is a useful addition. What do you all think? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.

2 1

Interest in integrating C decimal module into Python?
by Stefan Krah 22 Oct '09

22 Oct '09

Hi, as some of you know, recently I've released an arbitrary precision C library for decimal arithmetic together with a Python module: http://www.bytereef.org/libmpdec.html http://www.bytereef.org/fastdec.html Both the library and the module have been tested extensively. Fastdec currently differs from decimal.py in a couple of ways that could be fixed. The license is AGPL, but if there is interest in integrating it into Python I'd release it under a Python-compatible license. There have been several approaches towards getting C decimal arithmetic into Python: http://bugs.python.org/issue2486 Fastdec follows Raymond Hettinger's suggestion to provide wrappers for an independent C implementation. Arguments in favour of fastdec are: * Complete implementation of Mike Cowlishaw's specification * C library can be tested independently * Redundant arithmetic module for tests against decimal.py * Faster than Java BigDecimal * Compares relatively well in speed against gmpy To be clear, I would not want to _replace_ decimal.py. Rather I'd like to see a cdecimal module alongside decimal.py. I know that ultimately there should be a PEP for module inclusions. The purpose of this post is to gauge interest. Specifically: 1. Are you generally in favour of a C decimal module in Python? 2. Would fastdec - after achieving full decimal.py compatibility - be a serious candidate? 3. Could I use this list to settle a couple of questions, or would perhaps a Python developer be willing to work with me to make it compatible? I'm asking this to avoid doing work that would not find acceptance afterwards. Thanks, Stefan Krah

10 16

PEP 384
by Andrew Svetlov 22 Oct '09

22 Oct '09

Is there some public branch for this PEP? I like the idea and glad to look on implementation. Thanks.

2 1

Proposal: Moratorium on Python language changes
by Guido van Rossum 21 Oct '09

21 Oct '09

In the python-ideas list I've proposed a moratorium on language changes. It seems to be gaining momentum; if you want to have a say, come over. You can watch the discussion in the archives starting here: http://mail.python.org/pipermail/python-ideas/2009-October/006305.html . (Eventually I'll move the proposal over to python-dev, but I thought it only fair to give early warning to those who don't read python-ideas.) -- --Guido van Rossum (home page: http://www.python.org/~guido/)

1 0