Why isn't Python king of the hill?

[Martijn Faassen]

Geoffrey  Gerrietts <geoff at homegain.com> wrote:

[description of a large scale Python/Zope system]
> I think that answers the question without getting me in trouble?
> :)

That's clear, thank you.

I'm interested in moving business logic out of Zope myself to do
better MVC stuff, though another approach I'm taking is to to
create more general Python/Zope components that help make the 
business logic so short and explicit you don't need lots of code
for that anymore. :)

I'm not saying this approach is *working*; I'm just saying I tend to
think in that direction; split out abstractions from the application
into generic external components so the actual business logic can be 'smaller'.

One direction I'm currently thinking in is to split out workflow into
an explicit workflow component (a python-based workflow management
system, doing for workflow what relational database management systems do
for data). It's still a lot of vague ideas and nothing serious has been
hatched yet, though.

> Martijn writes (quoting me):
>> > I've looked at ZODB and ZEO some for my personal edification. If
>> > I'm not mistaken, they provide a great mechanism for object
>> > persistence with key-based retrieval. My evaluation suggested
>> > that ZODB and ZEO were not very good at handling arbitrary
>> > retrieval, or even retrieval at the level of a SQL select
>> > statement.
>> 
>> What do you mean with 'arbitrary retrieval'?

> I'm meaning something more along the lines of a real indexing and
> retrieval engine here. A query might be something like "I want
> all objects that have the word 'spam' in any of their
> attributes". This compares to key-based retrieval which says "I
> want the object with the key 'spam'" or the SQL-style retrieval
> which says "I want the objects that contain the word 'spam' in
> the column 'eggs' in the table 'bacon'."

Ah, a powerful query system, then. Difficult, too. Could you give
a real life example of an actual arbitrary retrieval system? It's
not full-text-search search, right? (the ZCatalog I believe already
can do something like that, on multiple attributes). 

> Our application doesn't require arbitrary retrieval in its
> current incarnation.

[numbers with a point in there somewhere snipped :)]

> To answer an un-quoted question, fixed point or decimal math is
> required because dollars are involved. Binary FP's approximations
> show up in odd places when you're doing decimal math, which makes
> people uncomfortable when dealing with money.

Okay, I figured.

> Martijn writes (quoting me):
>> > There's a certain draw to being able to take a single webapp and
>> > deploy it on any of a dozen different app servers that know how
>> > to handle the content, though it's small.
>> 
>> What do you mean by this, exactly? Are you referring to the whole
>> Enterprise JavaBeans framework (sorry if I confuse any of the
>> terminology here) that's been implemented by various vendors? I've
>> always been skeptical about how well that actually works, but here
>> I'm skeptical without any actual real knowledge. :)

> I can't speak to how easy it is to move from one app server to
> another. I /can/ speak for how much of the "hard work" is done
> for you by some of these enterprise containers -- a whole lot!

Could you go into some more detail on this? I've heard this more often,
but I'm still not entirely clear on *what* work is done. :)

> I'm not sure how Python and the Python community as it currently
> exists could actually /do/ the servlet thing. It's an API, but
> it's also a specification of a standard; Sun can get away with
> declaring The Standard because they have such tight control over
> Java. Python's more open community proves something of a weakness
> here -- Zope is "standard" in a de facto sense, but everyone who
> sets out to develop a templating language or a content server or
> an app server sets out to do it in their own way, with their own
> set of interfaces in mind. That means moving between servers
> requires a layer of adaptor code in the best case, or a total
> rewrite in worst.

I'm not interested in promoting a standard, I'm just interested in 
stealing ideas for Python-based APIs. I'm not interested in standardizing
these APIs. :)

I think Zope's new page template language (TAL and such) has some potential
to become something more close to a standard for templating XHTML, though
there will always be other systems, of course. 

Zope itself is slowly moving into the direction to be a server for 
simple Python objects, with (interface) adaptor layers on top to actually
use the object in the Zope framework (and publish it). This will hopefully
make it easier to exchange code with other Python software.

This all an aside; now spill the beans about what makes these Java
APIs so great! :)

> Martijn writes (quoting me):
>> > But the biggest draw is that someone who's proficient with
>> > Java is not too hard to find.
>> 
>> Are you sure about this? I mean, sure, lots of people are being
>> cranked out that can use Java to a certain extent, but is it
>> really true that it's not hard to find someone *good* at it?
>> 
>> > Someone who's proficient with Python is considerably harder to
>> > lay hands on.
>> 
>> But I'd think that anyone who's actually good at Java would be 
>> retrainable to use Python in a matter of days.

> Java programmers are more numerous than Python programmers.

No contest. They're being cranked out, though, so the average quality
is probably lower than that of still more-or-less self selected Python
programmers. But there are *so* many Java programmers it's definitely
more easy to find those.

[snip]
> I agree that a person could be retrained to use Python, and could
> learn Zope. Python's not a terribly difficult language to learn,
> and it's a language that keeps on giving the longer you use it.
> But I do think that it would be difficult for someone hired as a
> consultant to walk in cold and learn enough Python and Zope to
> make sense of how the app worked.

Are you sure that works for Java (or *any* language) though? :)
Wouldn't any significant application be non-trivial to figure out
for someone new to it, or do the standard APIs help so much?

[snip people coming up to speed easily]
> On the other hand, if the site had been less successful, and the
> company could not afford to bring on full-time staff and pay to
> have them trained -- if the company needed to hire a couple
> contract programmers to come in and change a few things, add a
> few things -- the company would have looked a very long time for
> someone proficient enough in Zope and Python to come in cold and
> make the changes they needed.

Agreed, I can see this concern, especially with Zope; not so much with
Python -- any decent programmer with some OO experience can pick up enough
Python in no time.

[snip]
> If the app were done in Java, it would be a lot easier to hire
> people who knew at least a little Java and could find their way
> around the servlet API and JSP, whether as permanent employees or
> temporary contractors. That's just because of the demographics.

> Meanwhile, it's also a lot easier -- at least in our area -- to
> send your employees off to a three day class on Java or JSP and
> servlets, and it's easier to learn all three topics from any
> number of widely-published, well-written books. Zope training
> involved flying Matt out here from Digital Creations, and the
> documentation is scarce.

I think we need to separate comparing Zope with the Java-style
solutions and Python to Java. I think there can be a good case made
that switching to Python is not that big a deal. Learning about
Zope is a whole different story, however. It's far easier than it
used to be, on the documentation side; community knowledge has also
increased. But your concerns with Zope make sense.

[snip scaling java versus scaling python; the jury is mostly out]

> Martijn writes:
>> Any larger app will have to grow and change over time, so I
>> consider this as well for my Zope/Python/relational database
>> apps, which is why I'm interested in your ideas about what
>> makes Java more suitable for these purposes. Not so I can go
>> and use Java, but for stealing some ideas. :)

> Probably the sanest thing either of us could be doing in this
> discussion. :)

> Let's see if we can come up with a short list -- I don't think
> these belong in the PEPs, really, maybe just ideas for people's
> projects?

Yup. Note that Zope is onto entity/session beans or whatever bizarro
terminology Java uses, though this is only in the exploration stages.
The proposed terms for Zope components are Content/Application/Presentation.

Here's a link into a 'half-baked wiki' (aren't they always?) discussing
this:

http://www.zope.org/Members/michel/Components/Components/NewReligion

> Maybe the first idea is some kind of formalized
> community that would provide support for these kinds of
> discussions? I don't know.

A 'Python generic application server APIs' mailing list? :) 
I'm not sure if that'd work.
 
> But the ideas.

> It would be very good if you could take an object, instantiate it
> in the context of a container, and basically have proxies
> propagated to all other applications that have instantiated that
> container.

Note that XML-RPC sort-of-almost does something like this; it looks
like proxies, at least. There are actually several light-weight
RPC mechanisms for Python that use proxy-objects like this
(and I heard the now-cancelled Ultima Online 2 project rolled their 
own).

Or what do you mean by 'other applications that have instantiated that
container'? Presumably they've instantiated a *proxy* of the container?

> Let's put it a different way, more abstract and less tied to
> implementation. It would be very good if you could write an
> object that conforms to a reasonable interface, and in so doing
> "gain" the ability to use the object in a distributed fashion.

Have you seen the interfaces PEP? The Python community is slowly
moving to supporting the notion of explicit interfaces (apart from
classes), with run-time querying and checking (does this object support
the 'foo' interface?).

> Currently, you've got to do it all by hand, and you can't count
> on the way you do it one time being re-usable the next time.
> There's no /standard/ way of doing it, and there's more than one
> way to do it. That's treading into Perl's territory, there....

Agreed that there's a need for this. Once we have interfaces,
remote querying and so on is only a small step away, of course.

> Here's something I like about the servlet containers (app
> servers) that I haven't seen in Zope yet (maybe it's there?):
> managed sessions via cookies OR URL rewriting. Some of the
> servlet containers even support persistent sessions and sessions
> shared across multiple app servers. When you have fault-tolerance
> at that level of the application, that's powerful juju.
[snip]

Check out 'Core Session Tracking' (not yet in the Zope core, they're but
working on it). I think it may come close to supporting all of 
these, especially with ZEO.

http://www.zope.org/Members/mcdonc/Products/CoreSessionTracking

(see the release information, near the bottom for a description.
It also has extensive docs that you'll see when you install it)
 
[snip]
> Here's another one, one we can beat Java to the punch on: it
> would be nice if you could do object storage and retrieval into a
> relational database using a simple management framework. The
> upcoming Java Data Objects spec is an easy example to look at,
> but also consider looking at Apple/NeXT's Enterprise Objects
> Framework. I started working on something like this using
> metaclasses, and was reasonably successful, but it's a complex
> undertaking, and time consuming....

I believe there are various Zope projects looking into this 
(and has received quite a bit of attention recently). One is 
ZPatterns, something I never got into. Another recent project is
SmartObjects. Finally the Digital Creations folks are looking into
better relational database integration.

Lots of developments that haven't been fully baked yet, of course, but
they're sure being baked. :)

> Martijn writes (quoting me):
>> > For more modest ambitions, the factors that weigh heavy on me may
>> > seem irrelevant or even overweening. :)
>> 
>> Many people are using Python for rather less modest ambitions,
>> however. 
>> I'm certainly planning to. :)

> And I think that HomeGain is using Python in a pretty significant
> application, too. We've got a lot of traffic and a whole bunch of
> data. It can work! It just doesn't seem to have all the tools to
> facilitate it, and it's not nearly as ubiquitous as other
> solutions.

It's definitely not as ubiquitous, but the tools are being worked on.
And in some areas Python may be further along than the Java
competition, due to its flexible nature. ZEO developments are very
interesting, for instance. While Java certainly has object database, I
imagine they're not as transparent in use as the ZODB can be.

Thanks for your response!

Martijn
-- 
History of the 20th Century: WW1, WW2, WW3?
No, WWW -- Could we be going in the right direction?