[Tutor] Low level socket and threading code in python in HUGEwebsites -

anil maran anilmrn at yahoo.com
Fri Aug 18 10:27:53 CEST 2006


thanks alan it is very enlightening
can one of you guys who have experience building sites
such as yahoo mail and stuff, explain what parts of a
webserver needs to be multithreaded
thanks

--- Alan Gauld <alan.gauld at freenet.co.uk> wrote:

> Anil,
> 
> I don't know if this is the kind of thuing they did
> on reddit, but a
> few years ago I was working on a large network
> management
> system (in C++ FWIW). It was monitoring a network of
> around
> 100,000 nodes.To ensure a timely flow of alarm
> traffic the server
> had 4 network cards and the code used multi
> threading and
> low level comms code to access those 4 IP addresses.
> 
> But the Sun Sparcstation I was using at the time
> onlky
> had one network card and IP address, so the code
> couldn't
> run without mmajor invasive debug statements,
> almosty
> rendering the threading inactive. So in practice I
> had to
> run my code on the server each time I worked on the
> threaded code.
> 
> I don;t know if thats the kind of issue he means but
> it might be
> an example of the kind of thing that a big web farm
> might
> experience.
> 
> On another project I had similar issues where my
> workstation
> didn't have enough RAM to compile (actually link) 
> the code
> (in C++ again), so although I could edit the code on
> my local
> machine I had to do the builds on the server (I only
> had
> 128M RAM, the server had 512M - and at the time
> 128M RAM cost over $400 - 8x16M modules!)
> 
> HTH,
> 
> Alan G.
> 
> ----- Original Message ----- 
> From: "anil maran" <anilmrn at yahoo.com>
> To: <tutor at python.org>
> Sent: Thursday, August 17, 2006 10:48 AM
> Subject: [Tutor] Low level socket and threading code
> in python in 
> HUGEwebsites -
> 
> 
> > What kind of low-level socket and threading code
> is necessary for a 
> > db
> >  intensive site such as yahoo. How do we use it
> from python? does
> >  anyone  have experience using it
> >
> >
> >
> http://redditblog.blogspot.com/2005/12/on-lisp.html
> > >From moving to python by the founders of reddit
> >
> >
> > Because of the low-level socket and threading code
> we had to write,
> > reddit would not run on my Mac, and I was always
> tethered to our
> > FreeBSD development server. Not being able to
> program offline is a
> > pain.
> >
> > I would imagine that they aren't just using a
> database.
> > most of the "low level socket stuff" I've seen is
> about setting
> > timeouts and doing Async IO.
> >
> >
> >
> > ---------------------------------
> > Do you Yahoo!?
> > Get on board. You're invited to try the new Yahoo!
> Mail Beta. 
> 
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


More information about the Tutor mailing list