On 01/05/2011 06:15 AM, Nick Coghlan wrote:
Perhaps it would focus the discussion if we picked one or two modules (in addition to __main__) as example cases.
functools comes in two pieces - partial and reduce are implemented in C in the _functools module, everything else is implemented in Python in functools itself. datetime, on the other hand, is a case of a pure acceleration module - if _datetime is available, it is expected to completely implement the datetime API.
_functools.partial and the classes in datetime all adopt the strategy of lying about their original location in __module__. This is probably the best available choice, as it makes pickling do the right thing.
The main downside with this approach is the way it confuses things like inspect.getsource (for datetime, it reports the pure Python versions as the source code for the C accelerated versions, for functools.partial it gives a technically accurate, but potentially misleading error message. If inspect could easily tell that the accelerated versions were in use, then it could handle the situation a bit more gracefully).
It seems Python tries pretty hard to hide external calls, (the cause of the confusion you mention above). It makes me wonder why python doesn't have an extern type (or types). Then instead of them being a source of confusion, they would be recognisable for what they are. They could have extra attributes to enable pickle and other tools to work in a nice way.