Why Johnny Can't Distribute Processing. WAS: Distributed computing using SOAP. What about speed ?

[Tim Daneliuk]

Skip Montanaro wrote:
> 
>     Tim> The short reason for this is that connection-oriented solutions
>     Tim> like RPCs create very "brittle" applications in highly distributed,
>     Tim> high-performance environments.  I am thus less than enthusiastic
>     Tim> about SOAP as a solution.
> 
> I'd suggest that most of the applications that use SOAP or XML-RPC are not
> "applications in highly distributed, high-performance environments".  Highly
> distributed, yes, if you're talking about over the Internet.  Most uses I'm
> aware of, however, are not meant to be high-performance the way you mean it.
> 

You're right, of course, but ...

Both distribution and scale are important in different situations.
For instance, even if performance is not much of an issue, if you have 
an n-tier application laying all over a network and you're using RPCs 
for fabric, any one node becoming unavailable halts the application.  

A lot of the reason the internet is getting away with these kinds of protocols 
right now is that the computational model isn't n-tier, or really even something
as limited as client-server.  While these architectures exist *behind* the web 
server, the distribution model across the internet is really more terminal-to-
server wherein the browser has more-or-less become a better VT100.  This is
a Good Thing, IMHO, because it recognizes that clients are not trustworthy,
cannot be made reliable, and so on.

The rub is that things like .NET from Microsoft and EJB from Sun propose
to change the computational architecture such that the unit of distribution
(across the internet AND behind the walls of an organization) will be an 
*object* (not a server) with RPC, or at least some transport with synchronous
semantics, as the underlying connective tissue.  If successful, these models
will drive us to very fine-grained distributed computing topologies which
will be slow, brittle, impossible to manage, very hard to recover after failure,
ad infinitum, ad nausem.

Microsoft saw this coming years ago - Well, their Enterprise Computing group did,
anyway.  Gates probably didn't even understand what they were talking about because
it wasn't written in BASIC ;) - and came up with the MTS/Viper tx queue management
system.  They also added async and transactional semantics to COM+.  The problem,
as always, with Microsoft is that: a) It is MS-centric and barely plays well with
others.  (.NET will play well with others only in the marketing literature.  In
reality, I suspect that anyone wanting to use the full .NET bag-o-trix will need
to do so primarily from Win32 platforms.) and b) Microsoft makes terrific
consumer-grade and end-user platforms, but their server technology is just
not up to running mission-critical high scale apps.  (Their ads say that they
do just that, but forget to mention the number of full time engineers that are
needed to crawl all over their enterprise servers to keep 'em alive.)  So,
Microsoft with arguably the best distribution harness in the business right now,
is out of the running because their underlying platform is simply not appropriate
for very larger problems and they don't run on other, more adult platforms.

Sun (and EJB) is, IMHO, out of the running because they are married to those
synchronous transport semantics.

So, here's what we need.  We need a platform-neutral transactional messaging and
queing service with features like:

1) Aynchronous APIs to the messaging and queuing manager
2) Platform-neutrality: From mainframes to desktops
3) Class-Of-Service selection
4) Per-event, selectable write through to backing store
5) Bindings for C and Java, and also the major scripting langs: Python, TCL, Perl, VBA
6) Remote management and recovery capability
7) Plugable security and encryption.

There are only two commercial products that come close to this, of which I
am aware.  BEA bought the rights to the old DEC Message Queue product, but
I dunno if they're doing anything with it any more.  And IBM, of course,
has MQ-Series, which has some of the above.

I keep thinking about designing such a beast myself, but the time to get
it done seems daunting.  But, it's one of the background tasks that runs
while I sleep, so you never know...                                                        

------------------------------------------------------------------------------
Tim Daneliuk
tundra at tundraware.com