python's threading has no "interrupt"?

[Dave Brueck]

Jay wrote:
> >So, if I understand this correctly, this amounted to a manual task switcher,
> >right? (as no more than one job was allowed to run concurrently). Was this
> >really the desired behavior? If each process halts while the others run and
> >does so only to prevent starvation in the other processes, wouldn't you get
> >more or less the same results by just using normal forking/threading wherein
> >the OS ensures that each process/thread gets a slice of the CPU pie? (IOW,
> >unless there was some other reason, it seems that the explicit yielding was
to
> >work around the constraints of the process model)
>
> Partially, the processes were sharing a (sometimes very large) common
> in-memory object web so forking seperate os level threads was not
> feasible.  Also, the server ran on multiple OS platforms so OS level
> tricks or threading mechanisms were not viable

Ok, but it should still be very doable with just threads (as opposed to forking
new processes - although forking might still work well with shared memory), and
preemptive threading is more the norm than not these days, at least on the OS's
that Python runs on - so having the OS schedule the threads doesn't require any
special "tricks", it just works.

> So... what was going wrong that warranted killing the process in the first
>
> >In practice the need for that is pretty rare, especially for a server,
> >the main case coming to mind being the execution of user-supplied code
(which
> >is pretty scary in and of itself!).
>
> Irt didn't happen that often but it was mostly due to the client
> disconneecting permaturely and when yuor server is handling heavy
> traffic, it does become a resource issue after awhile.

Having the handler detect and clean up by itself after a premature client
disconnect is both cleaner and results in _lower_ average resource usage
because it could be detected much more quickly and reliably than some watchdog
thread (this approach can work well on both poll-based and threaded connection
handling). You'd also get the benefit of more information in making the
decision: a watchdog cannot distinguish between slow progress and no progress.

>  Like I said in
> anoter post, there was also bookkeeping down during this time as well.

Again though, you get the scheduling "for free", so e.g. there's no problem
with having a bookkeeping thread in Python either.

-Dave