[Python-Dev] Parsing vs. lexing.

Barry A. Warsaw barry@python.org
Wed, 21 Aug 2002 15:59:42 -0400

>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

    GvR> Another note for Aahz: when it comes to scanning data that's
    GvR> not really a programming language, e.g. email messages, the
    GvR> words parsing, scanning, lexing and tokenizing are often used
    GvR> pretty much interchangeably.

True, although even stuff like email messages are defined by a formal
grammar, i.e. RFC 2822.  email.Generator of course doesn't strictly
use that grammar because it's trying to allow a much greater leniency
in its input than a language compiler would.  But note that approaches
like Emacs's mail-extr.el package do in fact try to do more strict
parsing based on the grammar.