Newbie needs to see a large project

[Dave Brueck]

On Tuesday 07 October 2003 11:00 am, Nowan wrote:
> John Fabiani wrote:
> > Hi everyone,
> > I have been checking Python recently and have presented what little I
> > know to mangement.  They have asked if there is any large business apps
> > (like an accounting system) using python and what the performance is
> > like.  We would be running on Linux with a large database (maybe postgres
> > or mysql) of 3-5 Gbytes.  We would be moving from a windows
> > platform(foxpro apps) to Linux (python).  About 50 users total.  Thanks
> > in advance....
>
> ================================
> Great question.
>
> I'm in the same situation.  I'm an enterprise database guy who is
> looking at building database-backed transactional web apps.  Stuff that
> might otherwise be done using ASP or JSP/Servlets.
>
> I need a database-driven web app that might support a few hundred users.
>
>  From the many thoughtful responses, I'm afraid that a lot of folks
> haven't built large apps, and haven't built sophisticated database apps.

I'd characterize a few hundred users as a small-to-barely-medium site, 
depending largely on what exactly they do on that site. Would you agree?

> 10,000 LOC is not even a medium size app.

LOC is misleading, although it's often tough to come up with a better metric. 
What you really need is a metric of application functionality. With 10k lines 
of Python you can get a ton of functionality, often enough to fall into the 
"medium-sized app" category. 

Using LOC though, 10k LOC Python ~= 70k-90k of Java, which again puts the app 
at least on the lower end of medium-sized apps - certainly into the 
non-trivial category and beyond the limited set of functionality you'd 
traditionally associate with10k LOC.	

It would be much better if we could come up with a way to measure return on 
investment, because Python and similar languages would measure up even better 
IMO (as it would take into account lower up-front dev costs, 
maintenance/bug-fixing costs, transfer-of-ownership costs, etc.).

> In a database-backed web site, the database is definitely not the
> bottleneck if you are using Oracle or another enterprise class product.
>     Unless your database design is very complex, it is the
> performance/design of your web/middleware.

It really is tough to speak in general terms because the performance really 
depends on what the web site _does_ as I'm sure you're aware. If it's just 
serving up mostly static pages then, yeah, the database will probably be a 
non-issue. A company I consult for has a Zope site with an Oracle backend 
that does statistical collection and reporting and management of a network - 
the DB machine is often pegged and the Zope machine is often close to idle - 
the DB schema isn't all that complex but the DB is most definitely the main 
bottleneck, and by a long shot.

> I'm building my little demo app.  I'm loving Python.  Python rocks.  Way
> cooler than Perl.  Way cooler than PHP.  But the question looms ahead...
> When I get finished, will it be fast enough to be a practical
> alternative to ASP or JSP/Servlets?

Others may disagree, but I don't see very many compelling reasons to go with 
ASP or JSP/servlets, especially if you expect the project to get large (with 
ASP you'll start out ok, but suffer if the project gets large, with JSP 
you'll suffer both now and then :) ), so I'd say Python is _always_ a 
practical alternative to them. 

PHP is a messy language, but can be pretty fast. I haven't used it enough to 
say how it would do for a large website, but have heard some horror stories 
about small feature addition jobs that turn into absolute death marches - 
don't know if that is the norm or what. 

Also, it doesn't hurt to have a good understanding of how much performance you 
really truly _need_ - if you have a fast DB on the backend and a decent cache 
in front, and if your site isn't in one of the top few percent of large sites 
out there, and if you app is closer to a traditional site than a web-enabled 
application, then sufficient performance may be less of an issue than, say, 
time to deployment or cost of new features/maintenance.

One last thought that has probably already crossed your mind: don't 
underestimate the blessing of living under the rule of Moore's Law! The 
performance of one Python server component in our network was in fact CPU 
bound (with network in 2nd place). I estimated that performance was still 
about 30-40% above what we actually needed at the time (for one thing, we had 
to put in two boxes just for redundancy purposes anyway), but the optimizer 
in me wanted to boost performance anyway. Well, that was a year ago - I never 
got around to it because of other things that actually did need work. Those 
computers have been moved to the testing rack and in their places are boxes 
with double the CPU power. Now I don't know if I'll ever get around to 
optimizing that code because the reason to do so is much less compelling (and 
therefore less interesting) now. What _is_ interesting though is that that 
component went from design to production in about a month and has been in 
production ever since - so the company got a lot of benefit by quick delivery 
and the performance problems have yet to materialize. Obviously this won't be 
true in all cases, but in this one that's how it worked out.

Just my two cents,
Dave