[Python-ideas] Objectively Quantifying Readability
steve at pearwood.info
Tue May 1 08:37:11 EDT 2018
On Tue, May 01, 2018 at 04:50:05AM +0000, Dan Sommers wrote:
> On Tue, 01 May 2018 10:42:53 +1000, Steven D'Aprano wrote:
> > - people are not good judges of readability;
> WTF? By definition, people are the *only* judge of readability.¹
We're discussing an actual study that attempted, with some reasonable
amount of success, to objectively measure readability without human
judgement (aside from the initial judgement on which factors to measure
and which to ignore). So given the the fact that there exists at least
one non-human objective measurement of readability which corresponds
reasonably well with human judgement, how do you reach the conclusion
that only humans can be the judge of readability?
Besides, even if we agreed that only people can do something, that
doesn't mean that they are necessarily *good at it*.
> I happen to be an excellent judge of whether a given block of code is
> readable to me.
In the same way that 93% of people say that they are an above-average
driver, I'm sure that most people think that they are an excellent judge
of readability. Including myself in *both* of those.
(My wife thinks I'm a crappy driver, but her standard of "just barely
adequate" is close to professional racecar drivers, so I don't let that
There are at least three ways that I can justify my statement, one
objective, and two anecdotal:
1. The correlation between judgements of readability from different
people is not very good: in the study discussed, it was about 0.6 or so.
A correlation of 0.5 is equivalent to having everyone agree on half the
samples and then rate the other half at random.
The paper states:
"This analysis seems to confirm the widely-held belief that
humans agree significantly on what readable code looks like,
but not to an overwhelming extent."
2. Anecdotally, we all know that many programmers are just awful.
And presumably most of them think they are writing readable code. (There
may be a small minority who are deliberately writing obfuscated code,
but I doubt that's a significant demographic.) Most of us have had the
experience of making code choices that are inexplicable and unreadable
when we come back to it.
3. Anecdotally, I have first hand experience with many people, including
programmers, making dramatically sub-optimal choices while declaring
that it is the most effective choice. To pick one example that applies
to coders, I have known many people who swear black and blue that they
work best with their editor configured to show code in a tiny, 8pt font,
and I've watched them peering closely at the screen struggling to read
the text and making typo after typo which they failed to notice.
In other words, people are often not even a great judge of what is
readable to *themselves*.
More information about the Python-ideas