Mailman 3 December 2006 - Python-Dev

LSB: pyc stability
by "Martin v. Löwis" 04 Dec '06

04 Dec '06

Some people (Robert Schweikert) requested byte-code stability at the LSB meeting: LSB should standardize a certain version of the byte code, and then future versions of Python should continue to support this byte code version. I explained that this is currently not supported, but would be technically possible; somebody would have to implement it. What do you think? Regards, Martin

4 5

Polling with Pending Calls?
by Tony Nelson 04 Dec '06

04 Dec '06

I think I have a need to handle *nix signals through polling in a library. It looks like chaining Pending Calls is almost the way to do it, but I see that doing so would make the interpreter edgy. The RPM library takes (steals) the signal handling away from its client application. It has good reason to be so paranoid, but it breaks the handling keyboard interrupts, especially if rpmlib is used in the normal way: opened at the beginning, various things are done by the app, closed at the end. If there is an extended period in the middle where no calls are made to rpmlib (say, in yum during the downloading of packages or package headers), then responst to a keyboard interrupt can be delayed for /minutes/! Yum is presently doing something awful to work around that issue. It is possible to poll rpmlib to find if there is a pending keyboard interrupt. Client applications could have such polls sprinkled throughout them. I think getting yum, for example, to do that might be politically difficult. I'm hoping to propose a patch to rpmlib's Python bindings to do the polling automagically. Looking at Python's normal signal handling, I see that Py_AddPendingCall() and Py_MakePendingCalls(), and PyEvel_EvalFrameEx()'s ticker check are how signals and other async events are done. I could imagine making rpmlib's Python bindings add a Pending Call when the library is loaded (or some such), and that Pending Call would make a quick check of rpmlib's caught signals flags and then call Py_AddPendingCall() on itself. It appears that this would work, and is almost the expected thing to do, but unfortunately it would cause the ticker check to think that Py_MakePendingCalls() had failed and needed to be called again ASAP, which would drastically slow down the interpreter. Is there a right way to get the Python interpreter to poll something, or should I look for another approach? [I hope this message doesn't spend too many days in the grey list limbo.] -- ____________________________________________________________________ TonyN.:' The Great Writ <mailto:tonynelson@georgeanelson.com> ' is no more. <http://www.georgeanelson.com/>

2 2

Virtual Python (was Re: Python and the Linux Standard Base (LSB))
by Mike Orr 04 Dec '06

04 Dec '06

This may be a bit too FAQ-ish for some people but I'll post it in case anybody benefits from the answer. On 11/30/06, Phillip J. Eby <pje(a)telecommunity.com> wrote: > At 02:46 PM 11/30/2006 -0800, Mike Orr wrote: > >Speaking of Virtual Python [1], I've heard some people recommending it > >as a general solution to the "this library breaks that other > >application" problem and "this app needs a different version of X > >library than that other app does". > > It was actually created to help people who don't have root access (e.g. in > shared web hosting accounts) get their "own" Python. It does work for > other things, but I haven't been especially recommending it for anything > besides that, since eggs take care of multi-version app/library support > quite nicely. > > Indeeed, the virtual-python approach is itself unnecessary now, as > easy_install has much better PYTHONPATH support now, than when > virtual-python was created. What does this last part mean? Virtual Python is still one of the four recommended ways to set up a custom install location in the EasyInstall docs. http://peak.telecommunity.com/DevCenter/EasyInstall#custom-installation-loc… The other approaches work fine for giving each user a private install dir, but don't address the case of the same user wanting different install dirs for different projects. For instance, when exploring Pylons or TurboGears which install a lot of packages I may not otherwise want, I create a Virtual Python for each of them. If I'm developing an application under Virtual Python, I can see at a glance which packages my project needs installed. I can't think of any other way except Virtual Python to do this. Another point. Setuptools seems to have Two Ways To Do Things regarding package activiation. easy_install puts the latest-installed egg version in its .pth file so it's automatically available via a naive "import". This happens to clutters sys.path with a more entries than some people desire. Meanwhile, programs can use pkg_resources to activate a package or version that may not already be in sys.path. Is this the Way Of The Future? Should people start using pkg_resources for all packages they import? (That would also allow one to remove little-used packages from easy-install.pth.) Finally, one can use ~/.pydistutils.cfg to specify an install location, but that again allows only one location per user. And in the case of root, it can't distinguish between manually-installed packages and OS-installed packages, which the admin might want in different directories (/usr/local/lib/python vs /usr/lib/pythonVERSION). This hasn't been an issue because OSes haven't been using easy_install or eggs, but it will be when they start doing so. Do the PYTHONPATH improvements make it possible to just put a directory on your PYTHONPATH and have Python process .pth files in it without using the site.addsitedir() hack? That would probably be my biggest wishlist item. -- Mike Orr <sluggoster(a)gmail.com>

1 0

Re: [Python-Dev] Polling with Pending Calls?
by Gustavo Carneiro 04 Dec '06

04 Dec '06

This patch may interest you: http://www.python.org/sf/1564547 Not sure it completely solves your case, but it's at least close to your problem. On 12/4/06, Tony Nelson <tonynelson(a)georgeanelson.com> wrote: > > I think I have a need to handle *nix signals through polling in a library. > It looks like chaining Pending Calls is almost the way to do it, but I see > that doing so would make the interpreter edgy. > > The RPM library takes (steals) the signal handling away from its client > application. It has good reason to be so paranoid, but it breaks the > handling keyboard interrupts, especially if rpmlib is used in the normal > way: opened at the beginning, various things are done by the app, closed > at the end. If there is an extended period in the middle where no calls > are made to rpmlib (say, in yum during the downloading of packages or > package headers), then responst to a keyboard interrupt can be delayed for > /minutes/! Yum is presently doing something awful to work around that > issue. > > It is possible to poll rpmlib to find if there is a pending keyboard > interrupt. Client applications could have such polls sprinkled throughout > them. I think getting yum, for example, to do that might be politically > difficult. I'm hoping to propose a patch to rpmlib's Python bindings to > do > the polling automagically. > > Looking at Python's normal signal handling, I see that Py_AddPendingCall() > and Py_MakePendingCalls(), and PyEvel_EvalFrameEx()'s ticker check are > how > signals and other async events are done. I could imagine making rpmlib's > Python bindings add a Pending Call when the library is loaded (or some > such), and that Pending Call would make a quick check of rpmlib's caught > signals flags and then call Py_AddPendingCall() on itself. It appears > that > this would work, and is almost the expected thing to do, but unfortunately > it would cause the ticker check to think that Py_MakePendingCalls() had > failed and needed to be called again ASAP, which would drastically slow > down the interpreter. > > Is there a right way to get the Python interpreter to poll something, or > should I look for another approach? > > [I hope this message doesn't spend too many days in the grey list limbo.] > -- > ____________________________________________________________________ > TonyN.:' The Great Writ <mailto:tonynelson@georgeanelson.com> > ' is no more. <http://www.georgeanelson.com/> > _______________________________________________ > Python-Dev mailing list > Python-Dev(a)python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/gjcarneiro%40gmail.com > -- Gustavo J. A. M. Carneiro "The universe is always one step beyond logic."

2 1

Python and LSB: Meeting Report
by "Martin v. Löwis" 04 Dec '06

04 Dec '06

We just had (the first day of) an LSB face-to-face meeting today [1], where Python was discussed. I started with presenting my slides, and stated what I see as main goals: - Allow Python scripts to run unmodified across Linux distributions - Optional: Allow extension modules to be used in binary form across distributions - Optional: Allow extension modules to be used in binary form across distributions During the discussion, Robert Schweikert proposed another goal, from the view of a proprietary software vendor: - Allow .pyc files to run unmodified across Linux distributions The discussion evolved around backwards compatibility. I'll start separate threads for these; here is the summary: - LSB attempts to guarantee backwards compatibility with previous versions (starting with 4.0; currently, 3.2 is targeted). Linux distributions implementing future LSB versions should be backwards-compatible with previous LSB versions. - Removal of features can only occur after three major LSB releases; this means all features will stay for about 6 years - If LSB integrated a specific version now, that version will have to be supported by vendors for those 6 years - given that the Python ABI changes frequently over time, this makes standardization of the ABI difficult - even for API (i.e. library interfaces), backwards-compatibility would be difficult: for those packages/classes/modules/functions that become part of the LSB API, stability would be desirable. Regards, Martin [1] http://www.freestandards.org/en/LSB_face-to-face_(December_2006) [2]http://www.freestandards.org/images/6/66/Lsb-f2f-200612-python.pdf

1 0

Makefile.pre.in Patch
by Hasan Diwan 04 Dec '06

04 Dec '06

The attached patch ensures that the $(DESTDIR) exists before installing the built binaries. It has been tested on Mac OS X. The patch is attached. -- Cheers, Hasan Diwan <hasan.diwan(a)gmail.com>

2 1

Re: [Python-Dev] Small tweak to tokenize.py?
by Phillip J. Eby 02 Dec '06

02 Dec '06

At 09:49 AM 11/30/2006 -0800, Guido van Rossum wrote: >I've got a small tweak to tokenize.py that I'd like to run by folks here. > >I'm working on a refactoring tool for Python 2.x-to-3.x conversion, >and my approach is to build a full parse tree with annotations that >show where the whitespace and comments go. I use the tokenize module >to scan the input. This is nearly perfect (I can render code from the >parse tree and it will be an exact match of the input) except for >continuation lines -- while the tokenize gives me pseudo-tokens for >comments and "ignored" newlines, it doesn't give me the backslashes at >all (while it does give me the newline following the backslash). The following routine will render a token stream, and it automatically restores the missing \'s. I don't know if it'll work with your patch, but perhaps you could use it instead of changing tokenize. For the documentation and examples, see: http://peak.telecommunity.com/DevCenter/scale.dsl#converting-tokens-back-to… def detokenize(tokens, indent=0): """Convert `tokens` iterable back to a string.""" out = []; add = out.append lr,lc,last = 0,0,'' baseindent = None for tok, val, (sr,sc), (er,ec), line in flatten_stmt(tokens): # Insert trailing line continuation and blanks for skipped lines lr = lr or sr # first line of input is first line of output if sr>lr: if last: if len(last)>lc: add(last[lc:]) lr+=1 if sr>lr: add(' '*indent + '\\\n'*(sr-lr)) # blank continuation lines lc = 0 # Re-indent first token on line if lc==0: if tok==INDENT: continue # we want to dedent first actual token else: curindent = len(line[:sc].expandtabs()) if baseindent is None and tok not in WHITESPACE: baseindent = curindent elif baseindent is not None and curindent>=baseindent: add(' ' * (curindent-baseindent)) if indent and tok not in (DEDENT, ENDMARKER, NL, NEWLINE): add(' ' * indent) # Not at start of line, handle intraline whitespace by retaining it elif sc>lc: add(line[lc:sc]) if val: add(val) lr,lc,last = er,ec,line return ''.join(out)

3 5

OT: World's oldest ritual discovered. Worshipped the python 70, 000 years ago
by Oleg Broytmann 02 Dec '06

02 Dec '06

http://www.apollon.uio.no/vis/art/2006_4/Artikler/python_english (-: Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd(a)phd.pp.ru Programmers don't die, they just GOSUB without RETURN.

5 5

fpectl: does a better implementation make sense?
by Giovanni Bajo 01 Dec '06

01 Dec '06

Hello, I spent my last couple of hourse reading several past threads about fpectl. If I'm correct 1) fpectl is scheduled for deletion in 2.6. 2) The biggest problem is that the C standard says that it's undefined to return from a SIGFPE handler. Thus, it's impossible to traps floating point exceptions and convert them to Python exceptions in a way that really works. 3) Moreover, the current implementation of PyFPE_* macros (turning on/off the traps + setjmp) are pretty slow, so they're off by default. Now, I would like Python to rause exceptions (FloatingPointError) whenever a Inf or NaN is computed or used in calculations (which to the best of my little understanding of 754 basically means that I want all FPU errors to be detected and handled). I am not arguing that this should be the default behaviour, I'm just saying that I would like if there was a way to enable this (even if only at Python compile time, in fact). I read that Tim Peters suggested several times to rewrite fpectl so that it does not use traps/signals at all, but just checks the FPU bits to see if there was a FPU error. Basically, the PyFPE BEGIN macro would clear the FPU bits, and the STOP macro would check for FPU errors and raise an appropriate exception if needed. Is this suggestion still valid or people changed their mind meanwhile? Would such a rewrite of fpectl (or a new module with a different name) be accepted? -- Giovanni Bajo

2 1

Weekly Python Patch/Bug Summary
by Kurt B. Kaiser 01 Dec '06

01 Dec '06

Patch / Bug Summary ___________________ Patches : 407 open ( +1) / 3484 closed ( +5) / 3891 total ( +6) Bugs : 936 open ( +5) / 6363 closed (+14) / 7299 total (+19) RFE : 246 open ( +1) / 244 closed ( +0) / 490 total ( +1) New / Reopened Patches ______________________ sys.id() and sys.intern() (2006-11-23) http://python.org/sf/1601678 opened by Georg Brandl clarify comparison return values (2006-11-24) http://python.org/sf/1602128 opened by Jim Jewett non-framework built python fails to define environ properly (2006-11-23) http://python.org/sf/1602133 opened by paul Suggest a textlist() method for ElementTree (2006-11-24) http://python.org/sf/1602189 opened by Raymond Hettinger subprocess: error redirecting i/o from non-console process (2006-11-27) http://python.org/sf/1603907 opened by Oren Tirosh Performance boost for array repeat (2006-11-28) CLOSED http://python.org/sf/1605020 opened by Mike Klaas Make Imap Error more helpful (2006-11-29) http://python.org/sf/1605192 opened by Thomas Guettler vendor-packages directory. (2005-09-22) http://python.org/sf/1298835 reopened by loewis Patches Closed ______________ __bool__ instead of __nonzero__ (2006-11-21) http://python.org/sf/1600346 closed by jackdied SRE engine do not release the GIL (2005-11-25) http://python.org/sf/1366311 closed by eric_noyau Implement lazy read for sockets (2003-12-04) http://python.org/sf/853963 closed by loewis Disable POSIX for certain/older NetBSD versions (2003-12-05) http://python.org/sf/854796 closed by loewis Performance boost for array repeat (2006-11-28) http://python.org/sf/1605020 closed by mklaas New / Reopened Bugs ___________________ using python extension(wxPython) in c (2006-11-23) CLOSED http://python.org/sf/1601607 opened by jolleydtan getopt Documentation improvement (2006-11-23) CLOSED http://python.org/sf/1601630 opened by Thomas Guettler Incorrect docs for bisect_left (2006-11-24) http://python.org/sf/1602378 opened by Daniel Eloff itemconfigure returns incorrect text property of text items (2006-11-25) http://python.org/sf/1602742 opened by Wojciech Mula wave module forgets to close file on exception (2006-11-26) http://python.org/sf/1603150 opened by amerinese wave module forgets to close file on exception (2006-11-26) CLOSED http://python.org/sf/1603246 opened by amerinese pstats module (profiler) doens't support unicode paths (2006-11-26) CLOSED http://python.org/sf/1603321 opened by Adal Chiriliuc def foo(((x))) segfault (2006-11-26) CLOSED http://python.org/sf/1603332 opened by Tony Lownds f=open fails with TypeError (2006-11-26) CLOSED http://python.org/sf/1603336 opened by Gibholio f=open fails with TypeError (2006-11-26) CLOSED http://python.org/sf/1603412 opened by Gibholio subprocess.py (py2.5) wrongly claims py2.2 compatibility (2006-11-27) http://python.org/sf/1603424 opened by Tim Wegener Python socket library confused by IPV6 notation in /etc/host (2006-11-27) http://python.org/sf/1603527 opened by Eric S. Raymond SaveConfigParser.write() doesn't quote %-Sign (2006-11-27) http://python.org/sf/1603688 opened by Rebecca Breu grammatical error in Python Library Reference::Tkinter (2006-11-27) CLOSED http://python.org/sf/1603789 opened by Gabriel M. Elder subprocess.Popen closes fds for sys.stdout or sys.stderr (2006-11-28) http://python.org/sf/1604851 opened by Nishkar Grover _CRT_SECURE_NO_DEPRECATE macro redefinition with VC++ 8 (2006-11-28) CLOSED http://python.org/sf/1604862 opened by William Fulton logging %(module)s reporting wrong modules (2006-11-29) http://python.org/sf/1605110 opened by Mad-Marty csv module broken for unicode (2006-11-30) http://python.org/sf/1606092 opened by JettLogic readline on popen3 file returns empty string before end (2006-11-30) http://python.org/sf/1606233 opened by Bill Wallace Bugs Closed ___________ utf_8_sig decode fails with buffer input (2006-11-23) http://python.org/sf/1601501 closed by doerwalter using python extension(wxPython) in c (2006-11-23) http://python.org/sf/1601607 closed by loewis getopt Documentation improvement (2006-11-23) http://python.org/sf/1601630 closed by gbrandl ctypes Structure allows recursive definition (2006-11-17) http://python.org/sf/1598620 closed by theller 2.4 & 2.5 can't create win installer on linux (2006-10-04) http://python.org/sf/1570417 closed by loewis wave module forgets to close file on exception (2006-11-26) http://python.org/sf/1603246 closed by gbrandl pstats module (profiler) doens't support unicode paths (2006-11-26) http://python.org/sf/1603321 closed by gbrandl def foo(((x))) segfault (2006-11-26) http://python.org/sf/1603332 closed by gbrandl f=open fails with TypeError (2006-11-26) http://python.org/sf/1603336 closed by gbrandl f=open fails with TypeError (2006-11-26) http://python.org/sf/1603412 closed by nnorwitz grammatical error in Python Library Reference::Tkinter (2006-11-27) http://python.org/sf/1603789 closed by gbrandl read() in windows stops on chr(26) (2006-11-13) http://python.org/sf/1595822 closed by sf-robot _ctypes built with GCC on AIX 5.3 fails with ld ffi error (2006-09-23) http://python.org/sf/1563807 closed by theller _CRT_SECURE_NO_DEPRECATE macro redefinition with VC++ 8 (2006-11-28) http://python.org/sf/1604862 closed by loewis "make install" for Python 2.4.4 not working properly (2006-10-19) http://python.org/sf/1580563 closed by sf-robot

1 0