[Python-Dev] status of development documentation

[Tim Peters]

[Neal Norwitz]
> This gives me an idea (ie, wild ass guess).  r39680 checked in on
> 2005-10-06 to speed up unicode charmap decoding.  Dunno if it's likely
> or not.  Gotta run, I'm headed east.  Good luck.

Nope, it's not calling any decoding functions at all on Windows, let
alone optimized ones ;-)

Who knows what the code is supposed to do?  (Sorry, but I don't want
to chase random patch guesses -- it's a wasteful approach to this
one.)

I'm getting a strong suspicion that I'm the only developer to _try_
building the trunk on WinXP since the AST merge was done, and that
something obscure is fundamentally broken with it on this box.  For
example, in tokenizer.c, these functions don't even exist on Windows
today (because an enclosing #ifdef says not to compile them):

error_ret
new_string
get_normal_name
get_coding_spec
check_coding_spec
check_bom
fp_readl
fp_setreadl
fp_getc
fp_ungetc
decoding_fgets
decoding_feof
buf_getc
buf_ungetc
buf_setreadl
translate_into_utf8
decode_str

OK, that's not quite true.  "Degenerate" forms of three of those
functions exist on Windows:

static char *
decoding_fgets(char *s, int size, struct tok_state *tok)
{
	return fgets(s, size, tok->fp);
}

static int
decoding_feof(struct tok_state *tok)
{
	return feof(tok->fp);
}

static const char *
decode_str(const char *str, struct tok_state *tok)
{
	return str;
}

In the simple failing test, that degenerate decode_str() is getting
called.  If the "fancy" decode_str() were being used instead, that one
_does_ call check_bom().  Why do we have two versions of these
functions?  Which set is supposed to be in use now?  What's the
meaning of "#ifdef PGEN" today?  Should it be true or false?

> Merry Christmas and Happy New Year everyone!

Same to you, Neal!