[OT] Syntax highlighting [was Re: Too much code - slicing]

[Seebs]

On 2010-09-20, Steven D'Aprano <steve at REMOVE-THIS-cybersource.com.au> wrote:
> On Sun, 19 Sep 2010 07:36:11 +0000, Seebs wrote:
>> No, but the syntax should be invisible.  When I read English, I don't
>> have to think about nouns and verbs and such unless something is very
>> badly written.

> That's almost certainly because you've been listening to, speaking, 
> reading and writing English since you were a small child, and the syntax 
> and grammar of English is buried deep in your brain.

Yes.  But I've been programming long enough that I seem to get similar
results in most languages pretty quickly.

> And you certainly do think about nouns and verbs, you just don't 
> *consciously* think about them.

Well, yes.  But it's conscious think time that's the limiting resource
for the most part -- so if I can avoid things that require conscious
thought, that frees up more for thinking about the problem.

> you will probably recognise "bloog" as the verb and "mobblet" as the 
> noun, even though you've almost certainly never seen those words before 
> and have no idea what they mean. But if I write this:

> "Susan is mobblet the blooged."

> you'll probably give a double-take. The words don't look right for 
> English grammar and syntax.

Well, actually, at that point I just assume you missed the capital
m on what is apparently a proper noun.  :)

> I've been reading, writing and thinking in Python for well over a decade. 
> The syntax and grammar is almost entirely invisible to me too. No 
> surprise there -- they are relatively close to that of the human 
> languages I'm used to (English). But if I were a native Chinese or Arabic 
> speaker, I'd probably find Python much less "natural" and *would* need to 
> explicitly think about the syntax more.

That's a fascinating question.  I don't think that would be the case, though.
Or at least.  If you've used more than a couple of programming languages that
much, I wouldn't expect it to be the case.  I'm not a native speaker of
Chinese, but after a year in China, I stopped perceiving grammar and just
heard sentences.  (Sadly, I've mostly since lost the vocabulary, leaving
me with the annoyance of a language I can think in grammatically but can't
express much of anything in.)

>> I've never seen this.  I've seen things highlight comments and keywords
>> and operators and constants and identifiers differently.

> Exactly. Things are highlighted because of *what* they are, not because 
> of the syntax they use or because of the grammatical role they play.

Hmm, interesting point.  e.g., a function name is likely to be highlighted
the same whether I'm calling it or referring to it as an object.  (I'm
very new to Python, so I'm not 100% sure functions are a kind of an object,
but I seem to recall they were.)

I guess that's a point; "syntax coloring" is perhaps not the right word
either for what they do.

> In a Python expression like:

> y = none or None

> an editor might colour "None" green because it's a known keyword, but 
> "none" black because it's a variable. If you change the syntax:

> y = None if [none][0] is None else {None: none}[None]

> the colours remain the same. None is coloured green not because of 
> *where* it is in the syntax tree, but because of *what* it is. Calling 
> this "syntax highlighting" is misleading, or at least incomplete.

This strikes me as correct.  But it's not exactly semantics, either.
It's... I dunno what to call it.

>> Eww.  (I had not yet gotten to the point of finding out that whether
>> something was "built-in" or not substantially affected its semantics.)

> In some languages, built-in functions truly are special, e.g. they are 
> reserved words. That's not the case for Python. Nevertheless, the editors 
> I've used treat built-ins as "pseudo-reserved words" and colourise them.

Interesting.  I wonder why.  I guess just because if you meant to name
a variable with one of those words, maybe you'd want the reminder.

>> Punctuation is very different from highlighting, IMHO.  That said, I
>> find punctuation very effective at being small and discrete, clearly not
>> words, and easy to pick out.  Color cues are not nearly as good at being
>> inobtrusive but automatically parsed.

> Well that surely depends on the colour scheme you have.

Only partially.  The big thing, I think, is that punctuation is separate
things next to words, not attributes of words.  Come to think of it,
that may be why I sometimes like to see keywords and other times punctuation.
I like {} better than do/end, for the same reason I prefer parentheticals
in English to something like:

	And this is a digression contrived end digression example.

I much prefer:
	And this is a (contrived) example.

> To my eyes, the feature of syntax highlighting that alone makes it 
> worthwhile, its killer feature, is that I can set comments and docstrings 
> to grey. When I'm scanning code, being able to slide my eyes over greyed-
> out comments and docstrings and ignore them with essentially zero effort 
> is a huge help. That's the thing I most miss, more than anything else, 
> when using a dumb editor.

That makes some sense.  In sh/python/Ruby/lua, I don't have any troubles
with it because the comment mechanism is fairly unambiguous.  I'm fine in
C as long as people remember the * on the left hand side of long comments.
Omit that, and I get fussy.  :)

> Just because nobody has done it yet doesn't mean that some sufficiently 
> intelligent software in the future couldn't do it :)

True.

It raises a curious question.  Imagine that you had the option of having
color highlighting to show precedence and/or grouping in complicated
expressions.  Would that be better or worse than parentheses?

For instance, consider the classic:

	x + y * z

In many programming languages, this is equivalent to:

	x + (y * z)

But would it be clearer to just have the unpunctuated text, with the "y * z"
in, say, a slightly lighter or darker shade?  I don't *think* so, but I'm
honestly not totally sure.

-s
-- 
Copyright 2010, all wrongs reversed.  Peter Seebach / usenet-nospam at seebs.net
http://www.seebs.net/log/ <-- lawsuits, religion, and funny pictures
http://en.wikipedia.org/wiki/Fair_Game_(Scientology) <-- get educated!
I am not speaking for my employer, although they do rent some of my opinions.