event loop problem

[Alex Martelli]

"Geoff Talvola" <gtalvola at nameconnector.com> wrote in message
news:mailman.972399545.29613.python-list at python.org...
    [snip]
> > > design flaw in NT that any one application can cause other
applications to
> > > hang if it's not processing its message loop.
> >
> > Only if those "other applications" *WANT* to hang; otherwise,
> > they would be using SendMessageTimeout (to guarantee termination
    [snip]
> The problem I have is that any one application that is *completely*
*unrelated*
> to another application can nevertheless cause that other application to
hang by

But, again, only if that "other application" *CHOOSES* to hang... and
please note the other application is NOT "completely unrelated" -- it
has explicitly asked the system "put me in touch with every other
process that has top-level windows, and wait infinitely for each of
them to answer".  That "any one application" DOES have top-level
windows, so it's in the set the "other application" has specifically
"infinitely-waited" about.

It may be silly to even provide a WAY to "wait forever" for something
to happen (when it can never be guaranteed that it will happen at all),
but all systems I know do provide such ways ("blocking system calls");
they're supposed to make life simpler than event-driven processing, I
think (for some obscure reason...).  Thus, doing so can hardly be
called a "design flaw" in a specific operating system.

> not processing its message loop.  From what you're saying, it's a bug in
those
> applications, not in Windows -- I can accept that.  However, it's not
exactly

I think the bug is in the application sending the "wait-forever" call
(SendMessage with broadcast).  It's explicitly putting itself at the
mercy of every other application in the Universe -- can that ever
make sense...?  It's a pity that SendMessage without the timeout is
hidden in the bowels of some libraries such as DDEML (I think it makes
those libraries unusable for quality applications, but some suppliers
of such applications clearly disagree with me!).

> obvious when you have accidentally created a hidden window and therefore
need to
> be processing messages.  I use Spy++ from Visual Studio to diagnose these
> things, but usually I don't realize I've created a hidden window until I
start
> noticing things like PythonWin not starting up :-)

Sure; I consider "not serving one's message-loop" a lesser bug (when
one doesn't _know_ there are hidden windows being created on one's
behalf by lower-level services...).


> > > Starting up PythonWin is
> > > another example of something you can't do if any program isn't
processing
> > > its messages.
> >
> > Really?  I hadn't noticed.  This is a bug in PythonWin, from
> > my POW -- it should NOT be doing a broadcast SendMessage without
> > a timeout (be it via DDEML or otherwise).  Have you submitted
> > this bug to ActiveState?
>
> I didn't know it *was* a bug, since I've seen so many other programs
behave the
> same way :-)

Well, it's a _widespread_ bug, I guess (it also affects some applications
I share responsibility for -- but let's not get into that:-).


> > > But a better solution is to tell win32com to run in free-threading
mode, in
> > > which case it doesn't create a hidden window and everything works fine
> >
> > No, but it DOES freely create threads!  Remember to protect
> > all access to shared-data if you choose this route.  Personally,
> > I think serving the message-loop is simpler.
>
> For a single-threaded Python program using ADO, there's no shared data to
worry

If that Python program exposes COM stuff, and it tells COM it's free
threaded, it's a risky gamble to call it "single-threaded".  _IT_ does
not explicitly start up more threads, BUT its code may be executed on
other threads (e.g. in response to COM events, if it hooks any).  And
I don't think the Python-interpreter's single global lock will save
your skin in that case -- the interaction level made "atomic" by that
lock is the single-Python-bytecode-instruction, which I think is too
semantically low (which is why Python multithreaded programs use their
own locking strategies).

> about.  And what if your program structure contains long-running
calculations
> based on data from your ADO database?  You have to contort your
calculations to
> insert PumpWaitingMessages() calls everywhere.  A single-threaded, non-GUI
> application shouldn't have to have a message loop, IMO.

I disagree.  _Particularly_ if that application "contains long-running
calculations", it *should* be checkpointing regularly, *and* responding
sensibly when the system tells it "we're about to go down, is that
all right?" -- which it does by sending it appropriate messages.  It's
even more important when the system has advanced power management: if
the application doesn't handle that, it has no way to tell the system
"no, DON'T switch to low-power-mode, I need all the CPU I can get for
my long-running calculations!", etc, etc.

Of course one doesn't explicitly call PumpWaitingMessages() -- but one
(IMHO) _does_ make sure a "globalobj.checkpoint()" is regularly called
for these purposes.  It costs nothing to 'pump' waiting messages in
that regularly-called-anyway checkpoint function.  It _would_ be nice
if there was a way to have the Python interpreter provide the "call
back to this function of mine regularly, please" functionality, in a
way, but then you'd be back to the problem of making sure your state
_is_ interruptible/checkpointable/re-entrant/...; by having _you_ call
the checkpointing function, it's far easy to guarantee you're ready
for checkpointing &c when you do call it (one of the key advantages
of event-driven programming over multithreaded-programming... it's
easier because you DO control WHEN you're done with an event and ready
to proceed...).


> In my experience, whether or not you're using free-threading mode, ADO
will
> create lots of threads and a hidden window for its own use anyway.  The
> difference is that in free-threading mode, the hidden window is in a
different
> thread and appears to have its messages processed automatically, whereas
if you
> use the default single-threaded apartment, the hidden window is created in
the
> main thread and you have to explicitly use PumpWaitingMessages everywhere.

I'll freely admit I have not looked at the issue of ADO "creating lots of
threads", and I do wonder where that extra 'hidden window' comes from (are
there apartment-threaded objects in the ADO/OleDb internals, not just
free-threaded ones...?).  But I wonder how you _ensure_ that none of your
code is ever executed on one of those other "lots of threads" -- maybe
by exposing no COM functionality whatsoever, including not hooking any
events.  Seems a rather high price to pay, in general, though it may be
OK for a lite/throw-away little thingy that doesn't need to receive
events or be externally-automatable anyway.


Alex