I think I have a need to handle *nix signals through polling in a library.
It looks like chaining Pending Calls is almost the way to do it, but I see
that doing so would make the interpreter edgy.
The RPM library takes (steals) the signal handling away from its client
application. It has good reason to be so paranoid, but it breaks the
handling keyboard interrupts, especially if rpmlib is used in the normal
way: opened at the beginning, various things are done by the app, closed
at the end. If there is an extended period in the middle where no calls
are made to rpmlib (say, in yum during the downloading of packages or
package headers), then responst to a keyboard interrupt can be delayed for
/minutes/! Yum is presently doing something awful to work around that
It is possible to poll rpmlib to find if there is a pending keyboard
interrupt. Client applications could have such polls sprinkled throughout
them. I think getting yum, for example, to do that might be politically
difficult. I'm hoping to propose a patch to rpmlib's Python bindings to do
the polling automagically.
Looking at Python's normal signal handling, I see that Py_AddPendingCall()
and Py_MakePendingCalls(), and PyEvel_EvalFrameEx()'s ticker check are how
signals and other async events are done. I could imagine making rpmlib's
Python bindings add a Pending Call when the library is loaded (or some
such), and that Pending Call would make a quick check of rpmlib's caught
signals flags and then call Py_AddPendingCall() on itself. It appears that
this would work, and is almost the expected thing to do, but unfortunately
it would cause the ticker check to think that Py_MakePendingCalls() had
failed and needed to be called again ASAP, which would drastically slow
down the interpreter.
Is there a right way to get the Python interpreter to poll something, or
should I look for another approach?
[I hope this message doesn't spend too many days in the grey list limbo.]
TonyN.:' The Great Writ <mailto:firstname.lastname@example.org>
' is no more. <http://www.georgeanelson.com/>
This may be a bit too FAQ-ish for some people but I'll post it in case
anybody benefits from the answer.
On 11/30/06, Phillip J. Eby <pje(a)telecommunity.com> wrote:
> At 02:46 PM 11/30/2006 -0800, Mike Orr wrote:
> >Speaking of Virtual Python , I've heard some people recommending it
> >as a general solution to the "this library breaks that other
> >application" problem and "this app needs a different version of X
> >library than that other app does".
> It was actually created to help people who don't have root access (e.g. in
> shared web hosting accounts) get their "own" Python. It does work for
> other things, but I haven't been especially recommending it for anything
> besides that, since eggs take care of multi-version app/library support
> quite nicely.
> Indeeed, the virtual-python approach is itself unnecessary now, as
> easy_install has much better PYTHONPATH support now, than when
> virtual-python was created.
What does this last part mean? Virtual Python is still one of the
four recommended ways to set up a custom install location in the
The other approaches work fine for giving each user a private install
dir, but don't address the case of the same user wanting different
install dirs for different projects. For instance, when exploring
Pylons or TurboGears which install a lot of packages I may not
otherwise want, I create a Virtual Python for each of them. If I'm
developing an application under Virtual Python, I can see at a glance
which packages my project needs installed. I can't think of any other
way except Virtual Python to do this.
Another point. Setuptools seems to have Two Ways To Do Things
regarding package activiation. easy_install puts the latest-installed
egg version in its .pth file so it's automatically available via a
naive "import". This happens to clutters sys.path with a more
entries than some people desire. Meanwhile, programs can use
pkg_resources to activate a package or version that may not already be
in sys.path. Is this the Way Of The Future? Should people start
using pkg_resources for all packages they import? (That would also
allow one to remove little-used packages from easy-install.pth.)
Finally, one can use ~/.pydistutils.cfg to specify an install
location, but that again allows only one location per user. And in the
case of root, it can't distinguish between manually-installed packages
and OS-installed packages, which the admin might want in different
directories (/usr/local/lib/python vs /usr/lib/pythonVERSION). This
hasn't been an issue because OSes haven't been using easy_install or
eggs, but it will be when they start doing so.
Do the PYTHONPATH improvements make it possible to just put a
directory on your PYTHONPATH and have Python process .pth files in it
without using the site.addsitedir() hack? That would probably be my
biggest wishlist item.
Mike Orr <sluggoster(a)gmail.com>
This patch may interest you: http://www.python.org/sf/1564547
Not sure it completely solves your case, but it's at least close to your
On 12/4/06, Tony Nelson <tonynelson(a)georgeanelson.com> wrote:
> I think I have a need to handle *nix signals through polling in a library.
> It looks like chaining Pending Calls is almost the way to do it, but I see
> that doing so would make the interpreter edgy.
> The RPM library takes (steals) the signal handling away from its client
> application. It has good reason to be so paranoid, but it breaks the
> handling keyboard interrupts, especially if rpmlib is used in the normal
> way: opened at the beginning, various things are done by the app, closed
> at the end. If there is an extended period in the middle where no calls
> are made to rpmlib (say, in yum during the downloading of packages or
> package headers), then responst to a keyboard interrupt can be delayed for
> /minutes/! Yum is presently doing something awful to work around that
> It is possible to poll rpmlib to find if there is a pending keyboard
> interrupt. Client applications could have such polls sprinkled throughout
> them. I think getting yum, for example, to do that might be politically
> difficult. I'm hoping to propose a patch to rpmlib's Python bindings to
> the polling automagically.
> Looking at Python's normal signal handling, I see that Py_AddPendingCall()
> and Py_MakePendingCalls(), and PyEvel_EvalFrameEx()'s ticker check are
> signals and other async events are done. I could imagine making rpmlib's
> Python bindings add a Pending Call when the library is loaded (or some
> such), and that Pending Call would make a quick check of rpmlib's caught
> signals flags and then call Py_AddPendingCall() on itself. It appears
> this would work, and is almost the expected thing to do, but unfortunately
> it would cause the ticker check to think that Py_MakePendingCalls() had
> failed and needed to be called again ASAP, which would drastically slow
> down the interpreter.
> Is there a right way to get the Python interpreter to poll something, or
> should I look for another approach?
> [I hope this message doesn't spend too many days in the grey list limbo.]
> TonyN.:' The Great Writ <mailto:email@example.com>
> ' is no more. <http://www.georgeanelson.com/>
> Python-Dev mailing list
Gustavo J. A. M. Carneiro
"The universe is always one step beyond logic."
We just had (the first day of) an LSB face-to-face meeting
today , where Python was discussed. I started with presenting
my slides, and stated what I see as main goals:
- Allow Python scripts to run unmodified across Linux distributions
- Optional: Allow extension modules to be used in binary form across
- Optional: Allow extension modules to be used in binary form across
During the discussion, Robert Schweikert proposed another goal, from
the view of a proprietary software vendor:
- Allow .pyc files to run unmodified across Linux distributions
The discussion evolved around backwards compatibility. I'll start
separate threads for these; here is the summary:
- LSB attempts to guarantee backwards compatibility with previous
versions (starting with 4.0; currently, 3.2 is targeted). Linux
distributions implementing future LSB versions should be
backwards-compatible with previous LSB versions.
- Removal of features can only occur after three major LSB releases;
this means all features will stay for about 6 years
- If LSB integrated a specific version now, that version will have
to be supported by vendors for those 6 years
- given that the Python ABI changes frequently over time, this
makes standardization of the ABI difficult
- even for API (i.e. library interfaces), backwards-compatibility
would be difficult: for those packages/classes/modules/functions
that become part of the LSB API, stability would be desirable.
At 09:49 AM 11/30/2006 -0800, Guido van Rossum wrote:
>I've got a small tweak to tokenize.py that I'd like to run by folks here.
>I'm working on a refactoring tool for Python 2.x-to-3.x conversion,
>and my approach is to build a full parse tree with annotations that
>show where the whitespace and comments go. I use the tokenize module
>to scan the input. This is nearly perfect (I can render code from the
>parse tree and it will be an exact match of the input) except for
>continuation lines -- while the tokenize gives me pseudo-tokens for
>comments and "ignored" newlines, it doesn't give me the backslashes at
>all (while it does give me the newline following the backslash).
The following routine will render a token stream, and it automatically
restores the missing \'s. I don't know if it'll work with your patch, but
perhaps you could use it instead of changing tokenize. For the
documentation and examples, see:
def detokenize(tokens, indent=0):
"""Convert `tokens` iterable back to a string."""
out = ; add = out.append
lr,lc,last = 0,0,''
baseindent = None
for tok, val, (sr,sc), (er,ec), line in flatten_stmt(tokens):
# Insert trailing line continuation and blanks for skipped lines
lr = lr or sr # first line of input is first line of output
add(' '*indent + '\\\n'*(sr-lr)) # blank continuation lines
lc = 0
# Re-indent first token on line
continue # we want to dedent first actual token
curindent = len(line[:sc].expandtabs())
if baseindent is None and tok not in WHITESPACE:
baseindent = curindent
elif baseindent is not None and curindent>=baseindent:
add(' ' * (curindent-baseindent))
if indent and tok not in (DEDENT, ENDMARKER, NL, NEWLINE):
add(' ' * indent)
# Not at start of line, handle intraline whitespace by retaining it
lr,lc,last = er,ec,line
I spent my last couple of hourse reading several past threads about fpectl. If
1) fpectl is scheduled for deletion in 2.6.
2) The biggest problem is that the C standard says that it's undefined to
return from a SIGFPE handler. Thus, it's impossible to traps floating point
exceptions and convert them to Python exceptions in a way that really works.
3) Moreover, the current implementation of PyFPE_* macros (turning on/off the
traps + setjmp) are pretty slow, so they're off by default.
Now, I would like Python to rause exceptions (FloatingPointError) whenever a
Inf or NaN is computed or used in calculations (which to the best of my little
understanding of 754 basically means that I want all FPU errors to be
detected and handled). I am not arguing that this should be the default
behaviour, I'm just saying that I would like if there was a way to enable this
(even if only at Python compile time, in fact).
I read that Tim Peters suggested several times to rewrite fpectl so that it
does not use traps/signals at all, but just checks the FPU bits to see if
there was a FPU error. Basically, the PyFPE BEGIN macro would clear the FPU
bits, and the STOP macro would check for FPU errors and raise an appropriate
exception if needed.
Is this suggestion still valid or people changed their mind meanwhile? Would
such a rewrite of fpectl (or a new module with a different name) be accepted?
On 09:34 am, jack.jansen(a)cwi.nl wrote:
>There's another standard place that is searched on MacOS: a per-user
>package directory ~/Library/Python/2.5/site-packages (the name "site-
>packages" is a misnomer, really). Standardising something here is
>less important than for vendor-packages (as the effect can easily be
>gotten by adding things to PYTHONPATH) but it has one advantage:
>distutils and such could be taught about it and provide an option to
>install either systemwide or for the current user only.
Yes, let's do that, please. I've long been annoyed that site.py sets up a local user installation directory, a very useful feature, but _only_ on OS X. I've long since promoted my personal hack to add a local user installation directory into a public project -- divmod's "Combinator" -- but it would definitely be preferable for Python to do something sane by default (and have setuptools et. al. support it).
I'd suggest using "~/.local/lib/pythonX.X/site-packages" for the "official" UNIX installation location, since it's what we're already using, and ~/.local seems like a convention being slowly adopted by GNOME and the like. I don't know the cultural equivalent in Windows - "%USERPROFILE%\Application Data\PythonXX" maybe?
It would be nice if site.py would do this in the same place as it sets up the "darwin"-specific path, and to set that path as a module global, so packaging tools could use "site.userinstdir" or something. Right now, if it's present, it's just some random entry on sys.path.