A lot of this reminds me of a story told by a programming instructor in the 70's, he submitted a FORTRAN program deck to the machine, the complier gave him a warning on a statement which read INTEGER misspelled, it than ran the program, but IGNORED the statement, even though it clearly understood what he meant, and got wrong answers because the compiler just used the default REAL type for the variable, which took him a while to figure out what the error was. (Computer time was limited enough then that you didn't want to just rerun fixing the typo that it pointed out). He was confused about how the program obviously knew what he meant by the message, but didn't process the program right.
An error like character (whatever) is not a quote (or is not a minus sign) seems similar. It is one thing to not recognize a funny character in the language, but to actually parse it well enough to give a message that says in effect, that may look like a quote to you, but I am not going to treat is as one, sounds perverse in the language. If we are going to go to the effort to detect that particular character, it makes more sense to make it actually DO the obvious thing. If not, the the current error seems fine, especially if we could include more details. An 'invalid character' message, that doesn't tell you WHICH character is invalid seems like it is holding back, If it included the bad character, or pointed to it, then the error becomes a lot more clear.