threading
Roy Smith
roy at panix.com
Wed Apr 9 10:44:38 EDT 2014
In article <mailman.9077.1397051723.18130.python-list at python.org>,
Chris Angelico <rosuav at gmail.com> wrote:
> For instance, the above code assumes that print() won't block. You
> might think "Duh, how can printing to the screen block?!?", but if
> your program's output is being piped into something else, it most
> certainly can :)
<nostalgia-mode>
Heh. One day, a long time ago, I had to investigate why our Vax-11/750
had crashed. I vaguely recollect being called at home on a weekend and
having to schlepp into work, but my memory may just be running in
auto-story-embelishment mode.
Turns out, the machine hadn't really crashed, but it was hung. The
console was a LA-120 (http://tinyurl.com/mljyegv), on which was printed
various log messages from time to time. It had run out of paper, which
was detected by the little out-of-paper microswitch, so it stopped
printing. When its input buffer got full, it sent a control-s, which
tells the thing on the other end of the serial line to stop sending.
Which of course caused the kernel tty driver output buffer to fill,
which eventually caused all the print statements by the various system
loggers to block, and eventually the whole mess ground to a halt.
I put in new paper. The printer proceeded to spit out several hours
worth of buffered log messages and the system picked up where it left
off.
</nostalgia-mode>
At Songza, we've been using gevent to do asynchronous I/O. It's an
interesting concept. Basically, you write your application code as you
normally would, using blocking I/O calls. Gevent then monkey-patches
the heck out of the Python library to intercept every call that could
possibly block and splice it into a asynchronous task scheduler
framework. The amazing thing is that it works. It let us reduce the
number of gunicorn worker processes we use by a factor of 6, and handle
the same amount of traffic.
Of course, monkey-patching is black magic, and sometimes we get hit by
really bizarre and difficult to track down bugs. But, to go back to my
"technology evolution" scale, gevent is somewhere between avant-garde
and what the kewl kids are using (version 1.0 was released in December),
so that shouldn't be too surprising.
More information about the Python-list
mailing list