Standard Deviation One-liner

[Steven D'Aprano]

On Fri, 03 Jun 2011 13:09:43 -0700, Raymond Hettinger wrote:

> On Jun 3, 10:55 am, Billy Mays <no... at nohow.com> wrote:
>> I'm trying to shorten a one-liner I have for calculating the standard
>> deviation of a list of numbers.  I have something so far, but I was
>> wondering if it could be made any shorter (without imports).
>>
>> Here's my function:
>>
>> a=lambda d:(sum((x-1.*sum(d)/len(d))**2 for x in
>> d)/(1.*(len(d)-1)))**.5
>>
>> The functions is invoked as follows:
>>
>>  >>> a([1,2,3,4])
>> 1.2909944487358056
> 
> Besides trying to do it one line, it is also interesting to write an
> one-pass version with incremental results:
> 
>   http://mathcentral.uregina.ca/QQ/database/QQ.09.06/h/murtaza2.html

I'm not convinced that's a good approach, although I haven't tried it. In 
general, the so-called "computational formula" for variance is optimized 
for pencil and paper calculations of small amounts of data, but is 
numerically unstable.

See 

http://www.johndcook.com/blog/2008/09/26/comparing-three-methods-of-
computing-standard-deviation/

http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance



I'll also take this opportunity to plug my experimental stats package, 
which includes coroutine-based running statistics, including standard 
deviation:

>>> s = stats.co.stdev()
>>> s.send(3)
nan
>>> s.send(2)
0.7071067811865476
>>> s.send(5)
1.5275252316519465
>>> s.send(5)
1.4999999999999998

The non-running calculation of stdev gives this:

>>> stats.stdev([3, 2, 5, 5])
1.5


http://pypi.python.org/pypi/stats/
http://code.google.com/p/pycalcstats/

Be warned that the version on Google Code is unstable, and currently 
broken.

Feedback is welcome!


-- 
Steven