[Tutor] Low level socket and threading code in python in HUGEwebsites -

wesley chun wescpy at gmail.com
Wed Aug 23 02:58:58 CEST 2006


On 8/18/06, anil maran <anilmrn at yahoo.com> wrote:
> thanks alan it is very enlightening
> can one of you guys who have experience building sites
> such as yahoo mail and stuff, explain what parts of a
> webserver needs to be multithreaded
>
> > > Because of the low-level socket and threading code
> > we had to write,
> > >
> > > I would imagine that they aren't just using a
> > database.
> > > most of the "low level socket stuff" I've seen is
> > about setting
> > > timeouts and doing Async IO.


even way back then, yahoo!mail was created on a similar platform as
most web-based applications are now:  apache (w/appropriate modules)
and a pool of child threads that captured incoming HTTP requests and
serviced them in an async manner.

we had std socket communication but no database access at all; none of
the code was threaded either... just apache.  it sounds relatively
simple, and in our case, it was from this architectural point-of-view.

the complexity was all in the infrastructure, i.e., managing all of
the servers on all of the networks, load-balancing, arranging the
outgoing and incoming SMTP, refreshing DNS MX records, all of the
users (old and new), user registration, and of course, the hard disks
where everyone's mail is stored. again, these things aren't special...
save for the mail-only stuff.  it's just a web app! :-)

HTH,
-- wesley
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
"Core Python Programming", Prentice Hall, (c)2007,2001
    http://corepython.com

wesley.j.chun :: wescpy-at-gmail.com
python training and technical consulting
cyberweb.consulting : silicon valley, ca
http://cyberwebconsulting.com


More information about the Tutor mailing list