Socket to me

[Irmen de Jong]

Mark Carter wrote:
> I'm trying to get my head around sockets/distributed computing - with
> some,
> albeit limited, 
> success. There was a thread on this board, dating from about beginning
> of April, which mentioned the following python-friendly technologies:
[...]

As the author of Pyro (one of the tech you mentioned) let me comment on your 
questions and thoughts.

> I'm a bit confused as to what they're all trying to achieve. I suspect
> that there's a fair bit of overlap. 

There probably is. Especially at the core - they all have to solve
the "how-to-talk-to-other-computers" issue. And isn't it the case,
that all things being equal, you usually wind up with the same
(type) of solution to the same (type) of problem?

> Would I be right in saying that 99
> times out of 100, these things are using sockets in order to
> communicate across machines?

I think you're correct and I also think that is because 99 out of 100 systems 
support BSD sockets and TCP/IP networking. Furthermore, communication on the 
internet is done using TCP/IP sockets. You can't have two computers connected 
by the internet talking to each other using shared memory, obviously :-)

> Am I right in thinking that the major considerations in writing
> servers
> is how one computer "knows" about another computer, and how to spawn
> threads so that multiple requests can be handled?

The "knowing eachother" issue is important in distributed computing. Pyro for 
instance has a Name Server that connects logical names to physical locations.

But I don't think it is a 'major consideration' in writing servers. Pyro's 
Name Server is a server on its own, and you can write other servers perfectly 
well without using it! The naming-location thing has nothing to do with the 
*server* specifically, it is merely a convenient means to find out where a 
specific server is. (in Pyro, that is).

Compare DNS. I can open http://216.239.41.99/ in my browser directly. I can 
also use the hostname-to-location server, DNS, to find out where Google is 
;-) Google's server works fine without DNS, it's only to make it more convenient.


> And would it be right to say that the purpose of "middleware" software
> is to handle all these details?

And more. Usually the goal of "middleware" is to abstract away as much as 
possible the gory, technical, complex details of the underlying communication 
mechanism. Also it could add a better Quality-Of-Service, for intance, 
messaging middleware often gives you guaranteed delivery, asynchronouos 
delivery, and what not, on top of the unreliable network layer.

Pyro's goal is to provide a distributed object environment in which you can 
forget about everything related to communication. I.e. it strives to 
interfere as little as possible with the normal, "single-program" Python 
style of programming. This makes it extremely easy to use because if you know 
how to use Python, you can use Pyro to make your system a distributed system, 
in a few lines of extra code.

> And once you know how to link in with this software, it basically
> removes the need for you to think about how all this glueing-together
> is supposed work, and lets you concentrate on coding what it actually
> is
> that you're trying to achieve?

Exactly, at least, that is what Pyro does.

> Would it also be fair to say that some projects seem to "kitchen-sink"
> the services they provide? I've noticed Twisted, for example, shows
> one
> how to construct a little echo server and client. But it seems to do a
> whole bunch of other stuff, whose purpose I have yet to understand.
> Don't
> get me wrong - I'm impressed by the cleverness of the guys that wrote
> Twisted.

Pyro is small and simple and will stay like that. It only includes the bare 
necessities; the core communication code, a Name Server, an Event Server for 
publish-subscribe type of communication, and that's it.
Pyro is about making your own Python objects remotely accessible, and nothing 
more really. Pyro is not about building a new HTTP-server.

> And following on from that, do I detect a certain element of "not
> invented here"? Perhaps a micro, rather than a monolithic approach
> to distributed computing would have been better - giving all of the 
> benefits of ... well, I'm sure we all know the drill about modular
> design.
> What we want is a layered cake, not alphabet soup.

Years ago when I started working on Pyro, there was NOTHING for Python that 
did whay I wanted to do: remote method invocation. So it was more of a 
necessity to "invent it here". Presently, there are a few other RMI packages 
for Python, but I still think none of them provides the same level of power, 
flexibility and ease-of-use that Pyro does.
Remember what you want to achieve; existing products just may not fit that 
description or may be designed with other goals in mind.

About modular design; for Pyro it is true that it is less modular than we 
would like it to be. It has grown a lot since the early versions and the way 
it is structured hasn't changed much. But to call it alphabet soup? You have 
to look at it to draw your own conclusion... ;-)

> Has anyone implemented a telnet daemon in python?

Twisted probably has ;-)

Regards,
--Irmen de Jong