
So, I was a bit irritated recently because there is no way to test for private name usage in a module's namespace. '__all__' is a good start, but it is somewhat poor for a few reasons: 1) it doesn't affect import modname; modname.someprivatename at all so you may be using private interfaces without having any way of knowing. 2) Because it doesn't actually affect code people write, it often doesn't get updated when new public interfaces get added. 3) until recently it didn't even affect help(module), so users don't even know they're using a private name. So, I wrote a tool to do the access checking. My goal is to have it used for testing (solving problems 1&2 *within* twisted, at least), not as something that you'd run your production server under. sandbox/foom/private.py. Currently, it just spits out warnings because that way I can actually run through all the twisted tests, but that's controllable as a flag. Ideally every module in twisted would define __all__ and we could run the entire testsuite with access control enabled without raising illegal access exceptions. To run the twisted tests with this, I temporarily stuck the following in twisted/__init__.py: import private private.privatizeModulesWithPrefix('twisted') Much of the output (attached at the end) is from the IMAP tests trying to access private bits in order to test them. I'm not sure exactly what to do about that -- tests _should_ sometimes be able to access private parts, but they should also be restricted, normally, because you want to test against the public interface as well. So I may have to create a way to annotate some tests as being able to access private attributes while disallowing it by default. Also, rebuild accesses private classes, and it is supposed to, so should be given special dispensation. Besides those, the other things that show up currently are: twisted.internet.abstract.isIPAddress from twisted.internet.iocpreactor.tcp (?) twisted.internet.protocol.ConsumerToProtocolAdapter from twisted.protocols.ftp (?) twisted.internet.threads._putResultInDeferred from twisted.enterprise.adbapi (_deferToThread) Some of those perhaps should be public (?), but _putResultInDeferred, probably should not. IMO these are bugs, either of omission in __all__ or access from an inappropriate place. I dunno which. Also, only 35 out of 559 modules in twisted use __all__. That's sad, and should be fixed. I'm sure if that was done, more of these kinds of things would show up. Here's the full list of accesses to private names (I took the output and sort|uniq'd it). '=> Yes' means access was granted based on the two modules living in the same package. '=> No' means it would have raised an exception if I had that enabled. <= private twisted.internet.abstract.isIPAddress from twisted.internet.iocpreactor.tcp (?): => No <= private twisted.internet.protocol.BaseProtocol from twisted.python.rebuild (latestClass): => No <= private twisted.internet.protocol.ConsumerToProtocolAdapter from twisted.protocols.ftp (?): => No <= private twisted.internet.threads._putResultInDeferred from twisted.enterprise.adbapi (_deferToThread): => No <= private twisted.mail.imap4.Command from twisted.mail.test.test_imap (append): => No <= private twisted.mail.imap4.DontQuoteMe from twisted.mail.test.test_imap (testQuoteAvoider): => No <= private twisted.mail.imap4.FileProducer from twisted.mail.test.test_imap (testFileProducer): => No <= private twisted.mail.imap4.MessageProducer from twisted.mail.test.test_imap (testMultipleMultiPart): => No <= private twisted.mail.imap4.MessageProducer from twisted.mail.test.test_imap (testSingleMultiPart): => No <= private twisted.mail.imap4.MessageProducer from twisted.mail.test.test_imap (testSinglePart): => No <= private twisted.mail.imap4.MessageSet from twisted.mail.test.test_imap (?): => No <= private twisted.mail.imap4.MessageSet from twisted.mail.test.test_imap (testSetFlags): => No <= private twisted.mail.imap4._FetchParser from twisted.mail.test.test_imap (testFetchParserBody): => No <= private twisted.mail.imap4._FetchParser from twisted.mail.test.test_imap (testFetchParserMacros): => No <= private twisted.mail.imap4._FetchParser from twisted.mail.test.test_imap (testFetchParserSimple): => No <= private twisted.mail.imap4._formatHeaders from twisted.mail.test.test_imap (testFetchHeaders): => No <= private twisted.mail.imap4._formatHeaders from twisted.mail.test.test_imap (testHeaderFormatter): => No <= private twisted.mail.imap4.collapseNestedLists from twisted.mail.test.test_imap (testFiles): => No <= private twisted.mail.imap4.collapseNestedLists from twisted.mail.test.test_imap (testQuoteAvoider): => No <= private twisted.mail.imap4.collapseStrings from twisted.mail.test.test_imap (testStringCollapser): => No <= private twisted.mail.imap4.decoder from twisted.mail.test.test_imap (testDecode): => No <= private twisted.mail.imap4.parseIdList from twisted.mail.test.test_imap (testIdListParser): => No <= private twisted.mail.imap4.parseNestedParens from twisted.mail.test.test_imap (_searchWork): => No <= private twisted.mail.imap4.parseNestedParens from twisted.mail.test.test_imap (testLiterals): => No <= private twisted.mail.imap4.parseNestedParens from twisted.mail.test.test_imap (testParenParser): => No <= private twisted.mail.imap4.splitQuoted from twisted.mail.test.test_imap (testQuotedSplitter): => No <= private twisted.mail.imap4.wildcardToRegexp from twisted.mail.test.test_imap (testWildcard): => No <= private twisted.mail.imap4.wildcardToRegexp from twisted.mail.test.test_imap (testWildcardNoDelim): => No <= private twisted.persisted.dirdbm.ALLOW_TWISTED_REBUILD from twisted.python.rebuild (rebuild): => No <= private twisted.web.xmlrpc.addIntrospection from twisted.web.test.test_xmlrpc (?): => No <= private twisted.internet.abstract.isIPAddress from twisted.internet.base (resolve): => Yes <= private twisted.internet.abstract.isIPAddress from twisted.internet.tcp (resolveAddress): => Yes <= private twisted.internet.abstract.isIPAddress from twisted.internet.udp (connect): => Yes <= private twisted.internet.abstract.isIPAddress from twisted.internet.udp (startListening): => Yes <= private twisted.internet.base.BaseConnector from twisted.internet.ssl (?): => Yes <= private twisted.internet.base.BaseConnector from twisted.internet.ssl (__init__): => Yes <= private twisted.internet.base.BaseConnector from twisted.internet.tcp (?): => Yes James