[Edu-sig] Why Python?

[David MacQuigg]

Bill Punch wrote:

> - concurrency. Others have commented on this already. Python is not 
> concurrent and will not likely be concurrent soon. Python does provide 
> threads, but those threads do not execute concurrently due to the 
> Global Interpreter Lock (GIL). You can learn threads, come to 
> understand them, but will not see the typical concurrent benefits in 
> Python. Now, if Python calls other processes externally, yes you can 
> get concurrency. You can even use external toolsets such as MPI to do 
> a better job. Lest you say this is unimportant, I think you should 
> reconsider. The world of computing has changed in the last 5 years. 
> You don't get faster cores, you just get more of them. The hexacores 
> are already being sold, with octocores soon to follow. Teaching early 
> programmers the importance of concurrency means that in the future 
> concurrency will be better utilized. Saying it isn't important dooms 
> younger programmers to not understanding the problem.

Speed is the issue here, and speed is definitely important for a small 
minority of applications.  You can get all the speed the machine is 
capable of, including cooking on all eight "octocores", by using C 
instead of Python at the bottlenecks in your program.  With C, you can 
bypass all the safety and convenience features that make Python slow 
(GIL, array limits, type checking, whatever) and just put the pedal to 
the metal.

Our freshmen start with a semester of C, then move on to Java, never 
using C again.  I have a lecture that I gave at the end of the semester, 
showing some beautiful Mandelbrot images.   
http://ece.arizona.edu/~edatools/ece175/Lecture/  It takes a couple of 
minutes to generate one high-resolution image, using the tightest Python 
code I can write.  Then I drop in some C (using weave.inline) for the 
critical loop.  The images are generated 200X faster!  My next step will 
be to see how much faster we can go with multiple threads.

Cython.org is working on a smooth integration of C with Python.  Do we 
still need a new language?  How would that language be better than 
Python + C?

> - list comprehensions. I love list comprehensions but hate the syntax. 
> It is overly confusing for a beginner. Other languages, for example 
> common lisp, have a much more elegant and consistent solution:
>
>    [x**2 for x in range(10) if x%2==0]
> vs
>    for x in range(10) collect x**2 when x%2==0
>
> A list comprehension is really an extension of the for iterator, why 
> not work with that syntax? Now we have three potential meanings for 
> something in [ ]: a list, an index and a list comprehension. Sure it 
> works, and you can even point me to the math foundation for it, but it 
> is uselessly complicated for a language structured for a beginner. 
> Extending the for iterator would have been the way to go IMHO.

I used to feel that list comprehensions were confusing, and better to 
just use a standard for-loop.  Then I read Michel Paul's Manifesto 
http://pykata.appspot.com/static/manifesto.py.txt, and I understood the 
elegance of the notation.  The first example above reads almost like it 
was standard set notation.

The problem was that I learned about list comprehensions from a text 
that only compared them to standard loops.  Save a few lines - no big 
deal.  Michel's presentation is much better.

-- Dave

> David MacQuigg wrote:
>> kirby urner wrote:
>>>> On Tue, Apr 13, 2010 at 10:30:49AM -0700, David MacQuigg wrote:
>>>>     
>>>
>>>>> That's not to say the 1% is unimportant.  Here we will find brilliant
>>>>> programmers working on sophisticated techniques to break large 
>>>>> problems
>>>>> into pieces that can be executed concurrently by hundreds of 
>>>>> processors.
>>>>> Each problem is very different, and we may find programs for circuit
>>>>> simulation using very different techniques than programs for weather
>>>>> prediction.  These programs will be run in an environment 
>>>>> controlled by
>>>>> Python.  The circuit designer or atmospheric scientist will not be
>>>>> concerned about the details of concurrency, as long as the result 
>>>>> is fast
>>>>> and accurate.
>>>>>       
>>>
>>> There's a school of thought out there that says operating systems are
>>> supposed to handle concurrency pretty well.  Some designs boot
>>> multiple Pythons as multiple processes and let the OS take care of
>>> concurrency issues.  Why reinvent the wheel and rely on internal
>>> threading?  Instead of multiple threads within Python, make each
>>> Python its own thread.
>>>
>>> These concurrently running Pythons are then trained to communicate in
>>> a loosely coupled fashion by reading and writing to centralized SQL
>>> tables, which double as audit trails.
>>>
>>> If a process dies, there might need to kill a process (zombie snake
>>> problem), and maybe the controlling process (like air traffic control)
>>> launches a new one -- depends on many factors.
>>>   
>>
>> As long as the coupling is loose, this kind of "concurrency" is 
>> easily handled by Python (either with threads, or separate 
>> processes).  The more challenging concurrency problems involve tight 
>> coupling, tighter than can be achieved with inter-process 
>> communications.   Think of a large system of equations, with 1000 
>> outputs depending on 1000 input variables.
>>
>> The challenge is in partitioning that problem into smaller problems 
>> that can be solved on separate processors, with manageable 
>> requirements relating to interprocess communication.  Strategies tend 
>> to be domain-specific.  If the domain is circuit design, the 
>> equations follow the modularity of the design, e.g. a collection of 
>> subcircuits, each with only one input and one output.  That strategy 
>> won't work in other domains, so we have no partition() function that 
>> will do it automatically, and (in my opinion) not much chance that 
>> some new language will come to the rescue.  For problems that really 
>> need partitioning, we need programmers that understand the problem 
>> domain.
>>
>> The question for teachers using Python is - Will there be some future 
>> "concurrency" language that is:
>> 1) So different that its features can't be unobtrusively merged into 
>> Python (not burdening those who don't need it).
>> 2) So easy that it will replace Python as a general-purpose language, 
>> even for those 99% that don't need concurrency.
>>
>> I'll keep an open mind.  Meanwhile, I've got to make a choice for 
>> PyKata.  We need to be ready for next semester.
>>