[Python-Dev] The C API and wide unicode support

[Guido van Rossum]

> > They're verboten in extension modules anyway, so I don't care.
> 
> They are not disallowed in extensions... don't know where you
> have that idea from.

Maybe because other macros are often disallowed in (3rd party)
extensions, the reason being that the macros dig in the internal
representation which isn't guaranteed to be binary compatible?  It
would make sense that the same rules applies to the Unicode macros in
3rd party extensions.

(I admit that these restrictions may be underdocumented.  Nevertheless
they were intended and I believe they were discussed.)

> Note that the name mangling is done to prevent an extension
> which uses Unicode in some way from loading if the interpreter
> and extension Unicode "width" doesn't match.
> 
> If we would allow this, extensions using the macros would cause
> memory corruption since they'd index differently. That's not only
> a potential cause for a seg fault, it's also a security risk.

If there was a way so that only extensions that use the macros or
APIs whose signature uses Py_UNICODE_TYPE would fail to load, that
would be better.  But I don't know how to enforce that.

> The name mangling does not provide a 100% bullet proof way
> of preventing this (an extension might use Py_UNICODE and
> the Unicode macros without touching any of the other C APIs),
> but it goes a long way in that direction.

Maybe it goes too far.

OTOH, Michael, is this really something you cannot live with?  Or is
it simply a surprise?

--Guido van Rossum (home page: http://www.python.org/~guido/)