[sapug] Plethora of jobs on SEEK for Python programmers in Adelaide.

stephen white steve at adam.com.au
Sat Mar 8 07:38:48 CET 2008


On 08/03/2008, at 11:20 AM, Chris Foote wrote:
>> - Dynamic languages suck for scaling up to real projects, even though
>> they're great for small scale RAD exploring.
>
> It depends what you mean by scaling.

I have to apologise for taking too much of your time with a small pot- 
stir aimed at maintaining my "demon spawn" reputation with Daryl.

> database update constraints - 10M number ranges, < 0.2s response  
> time to
> DNS queries regardless of load, and needed to withstand DOS attacks  
> (two

> On the other hand, if what you mean by scalability is that a language
> provides a good environment for >1M lines of code across >50  
> developers,

There is a third type of scaling, which is having one program running  
on multiple CPUs and across clusters of machines. Google's "map- 
reduce" is a one off "request-response" implementation of this, but  
it is possible to scale up from this to more general programming.

For example, Erlang is nearly parallelisable by default, and  
functional programming languages are automatically parallelisable  
except in the case where they have unclean side-effects.

Consider this sample:

	http://programming.nu/benwanu

The same technique can be used in Ruby and Python and Perl, but they  
would need bridges to the C code that performs the speed-critical  
section of Mandelbrot calculations. Because Nu is directly built on  
the underlying framework rather than bridging across, this is the  
actual code:

	http://code.neontology.com/gitweb.cgi?p=Nu.git;a=blob;f=examples/ 
Benwanu/objc/ 
Benwanu.m;h=fdd0ebf5c6b9a224b5324c161b58bc7bbd21802b;hb=HEAD

Not one bit of bridging to be seen. Nu calls the function directly,  
via Objective C's runtime. Any time any bit of Nu code is a bit too  
slow or needs something at a lower level, it can be directly dropped  
down a level without changing anything else.

This is in contrast to Ruby's C bridge, which effectively ruled out  
that option for me.

	http://www.onlamp.com/pub/a/onlamp/2004/11/18/extending_ruby.html

No thanks, that's just ridiculous.

> There are some good horizontal-scaling web systems written in Python.
> They easily scale for an ordinary "company web site".  Even a large

I agree with what you are saying for these systems, but these are web  
type workloads... a request comes in, the result is computed, and  
sent back. It's all "wham bam, thank you ma'am". I want to do  
computer vision algorithms, simulated annealing and run lenses  
distortion calculations over 100MB image files.

While C is fast enough, and Objective C is simple enough, I'm still  
lurking around dynamic languages because I want to type something and  
have a response (I'm the wham-bam!), without that thinking destroyer,  
the concentration disrupter, that awful edit-compile-run cycle.

> (list includes Akamai, Novell, Oxfam and many others for their primary
> web sites).

They are all web sites. They don't run large compute jobs on massive  
amounts of data using grids and clusters of machines.

Steve.

--
   steve at adam.com.au




More information about the sapug mailing list