[Python-ideas] Objectively Quantifying Readability

Steven D'Aprano steve at pearwood.info
Tue May 1 08:37:11 EDT 2018

On Tue, May 01, 2018 at 04:50:05AM +0000, Dan Sommers wrote:
> On Tue, 01 May 2018 10:42:53 +1000, Steven D'Aprano wrote:
> > - people are not good judges of readability;
> WTF?  By definition, people are the *only* judge of readability.¹

We're discussing an actual study that attempted, with some reasonable 
amount of success, to objectively measure readability without human 
judgement (aside from the initial judgement on which factors to measure 
and which to ignore). So given the the fact that there exists at least 
one non-human objective measurement of readability which corresponds 
reasonably well with human judgement, how do you reach the conclusion 
that only humans can be the judge of readability?

Besides, even if we agreed that only people can do something, that 
doesn't mean that they are necessarily *good at it*.

> I happen to be an excellent judge of whether a given block of code is
> readable to me.

In the same way that 93% of people say that they are an above-average 
driver, I'm sure that most people think that they are an excellent judge 
of readability. Including myself in *both* of those.

(My wife thinks I'm a crappy driver, but her standard of "just barely 
adequate" is close to professional racecar drivers, so I don't let that 
worry me.)


There are at least three ways that I can justify my statement, one 
objective, and two anecdotal:

1. The correlation between judgements of readability from different 
people is not very good: in the study discussed, it was about 0.6 or so. 
A correlation of 0.5 is equivalent to having everyone agree on half the 
samples and then rate the other half at random.

The paper states:

    "This analysis seems to confirm the widely-held belief that
    humans agree significantly on what readable code looks like,
    but not to an overwhelming extent."

2. Anecdotally, we all know that many programmers are just awful.


And presumably most of them think they are writing readable code. (There 
may be a small minority who are deliberately writing obfuscated code, 
but I doubt that's a significant demographic.) Most of us have had the 
experience of making code choices that are inexplicable and unreadable 
when we come back to it.


3. Anecdotally, I have first hand experience with many people, including 
programmers, making dramatically sub-optimal choices while declaring 
that it is the most effective choice. To pick one example that applies 
to coders, I have known many people who swear black and blue that they 
work best with their editor configured to show code in a tiny, 8pt font, 
and I've watched them peering closely at the screen struggling to read 
the text and making typo after typo which they failed to notice.

In other words, people are often not even a great judge of what is 
readable to *themselves*.


More information about the Python-ideas mailing list