I'm just starting to learn how to use Twisted. My first project is to write a simple XML-RPC server. I started with the demo, and that worked fine. I then added an import statement for the module I need. from openeye.oechem import * Now, this uses a shared library extension which was built for Python 2.2, and I'm running a CVS version of Python 2.3, so I usually get /usr/local/lib/python2.3/site-packages/openeye/oechem.py:5: RuntimeWarning: Python C API version mismatch for module _oechem: This Python has API version 1012, module _oechem has version 1011. import _oechem However, that didn't happen when I started my my simple server. I got no warning message. I started to worry that my file system wasn't properly saving files! I played around with the imports, and moved that import to be above the twisted ones, and it started reporting the warning. A bit of searching later and I found twisted/python/log.py which redefines (!) warnings.showwarning to dump the data to a logfile, which is initialized to a NullFile(), which does nothing. So the twisted code is throwing some (potentially useful) diagnostics. Doing a search I found the commit statement http://twistedmatrix.com/pipermail/twisted-commits/2002-October/ 003975.html
2) implemented a custom warning handler; now warnings look sexy. (the hackish overriding of warnings.showwarning is the recommended way to do so, according to the library reference.)
I'll grant that's what the docs say. However, why is the logfile for showwarnings initialized to NullFile, and not to sys.stderr? For that matter, I think it's better to delay replacement until startLogging is called. Hmm, and why is os.linesep used in the logging? The log file should have been opened in text mode, in which case "\n" will get translated to the proper os-dependent form. I started to change the code, so I could submit a patch. I got the code out of CVS and am reading the documentation at http://twistedmatrix.com/documents/howto/coding-standard It says that the main test suite builder is in 'twisted.test.test_all' That module doesn't exist (can't import it) nor is the appropriate file present anywhere. And the hyperlink from that page yields "No Such Resource File not found." It then says Acceptance tests are all automated by the bin/accepttests but that program is in admin/accepttests I ran that for my Mac OS X machine, with 'open' as the WEBBROWSER setting (where does the '-b' option go, since open doesn't block .. do I pass it to 'accepttests'? Yes.) and /bin/true as my IRCCLIENT, since I wouldn't know what to do with one even assuming I had one. Anyway, after a while it gave this error /------ |#### |#### Starting mail test. |#### Output should be one email (postmaster@foo.bar) |#### and one bounce (postmaster@no.such.domain). |#### \------ Running Command: './admin/../bin/mktap mail --domain foo.bar=dump --user postmaster=postmaster --pop 18110' option --pop not a unique prefix Usage: mktap mail [options] Options: -p, --pop3= Port to start the POP3 server on (0 to disable). [default: 8110] -S, --pop3s= Port to start the POP3-over-SSL server on (0 to disable). [default: 0] -s, --smtp= Port to start the SMTP server on (0 to disable). [default: 8025] -r, --relay= relay mail we do not know how to handle to this IP, using the given path as a queue directory -c, --certificate= Certificate file to use for SSL connections -d, --domain= generate an SMTP/POP3 virtual domain which saves to "path" --version --help Display this help and exit. -u, --user= add a user/password to the last specified domains --bounce-to-postmaster undelivered mails are sent to the postmaster This creates a mail.tap file that can be used by twistd. Running Command: './admin/../bin/twistd -f mail.tap' Failed to load application: [Errno 2] No such file or directory: 'mail.tap' Traceback (most recent call last): File "./admin/accepttests", line 271, in ? runAllTests() File "./admin/accepttests", line 247, in runAllTests runMailTest() File "./admin/accepttests", line 204, in runMailTest s = smtplib.SMTP('127.0.0.1', 18026) File "/usr/local/lib/python2.3/smtplib.py", line 240, in __init__ (code, msg) = self.connect(host, port) File "/usr/local/lib/python2.3/smtplib.py", line 302, in connect raise socket.error, msg socket.error: (61, 'Connection refused') Hmmm.... Is 'admin/runtests' the correct script to run? I ran that, and got the following output ======================================================================== ======= FAILURE: testReadLimit (twisted.test.test_policies.ThrottlingTestCase) ------------------------------------------------------------------------ ------- Traceback (most recent call last): File "/Users/dalke/cvses/Twisted/twisted/test/test_policies.py", line 193, in testReadLimit self.assertEquals(tServer.readThisSecond, 0) FailTest: 20 != 0 ======================================================================== ======= FAILURE: testWriteLimit (twisted.test.test_policies.ThrottlingTestCase) ------------------------------------------------------------------------ ------- Traceback (most recent call last): File "/Users/dalke/cvses/Twisted/twisted/test/test_policies.py", line 152, in testWriteLimit self.assert_(abs(p.wrappedProtocol.paused - now - 1.0) < 0.1) FailTest ======================================================================== ======= FAILURE: testPackages (twisted.test.test_setup.CheckingPackagesTestCase) ------------------------------------------------------------------------ ------- Traceback (most recent call last): File "/Users/dalke/cvses/Twisted/twisted/test/test_setup.py", line 44, in testPackages self.failUnlessEqual(l, []) FailTest: ['twisted.bugs', 'twisted.coils', 'twisted.eco', 'twisted.enterprise.userRequests', 'twisted.forum', 'twisted.inetd', 'twisted.issues', 'twisted.issues.ui', 'twisted.issues.ui.templates', 'twisted.lumberjack', 'twisted.metrics', 'twisted.pretzel', 'twisted.protocols.ldap', 'twisted.reality', 'twisted.reality.ui', 'twisted.secsh', 'twisted.test.interactive', 'twisted.web.blog', 'twisted.words.ui', 'twisted.words.ui.gateways'] != [] ======================================================================== ======= FAILURE: testFailing (twisted.test.test_tcp.LoopbackTestCase) ------------------------------------------------------------------------ ------- Traceback (most recent call last): File "/Users/dalke/cvses/Twisted/twisted/test/test_tcp.py", line 171, in testFailing self.assert_(time.time() - start < 0.1) FailTest ======================================================================== ======= ERROR: testUnicodeTolerance (twisted.test.test_xml.MicroDOMTest) ------------------------------------------------------------------------ ------- Traceback (most recent call last): File "/Users/dalke/cvses/Twisted/twisted/trial/unittest.py", line 165, in runOneTest method(testCase) File "/Users/dalke/cvses/Twisted/twisted/test/test_xml.py", line 326, in testUnicodeTolerance ud = microdom.parseString(s.encode('UTF-16')) File "./twisted/web/microdom.py", line 644, in parseString File "./twisted/web/microdom.py", line 609, in parse File "./twisted/protocols/sux.py", line 155, in dataReceived File "./twisted/protocols/sux.py", line 176, in do_begin File "./twisted/protocols/sux.py", line 108, in _parseError ParseError: <xmlfile />:1:1: First char of document ['\xfe'] wasn't < ======================================================================== ======= SKIPPED: test_modules (twisted.test.test_doc.DocCoverage) ------------------------------------------------------------------------ ------- Remove this line when you feel like writing docstrings. ------------------------------------------------------------------------ ------- Ran 494 tests in 227.667s FAILED (failures=4, errors=1, skips=1) Could not import twisted.test.test_conch: No module named Crypto.Cipher So I tried to rewrite the code given that it looks like the self-tests run reasonably well, but then ran into complications with this bit of code # Make sure we have some basic logging setup. This only works in cpython. try: logOwner except NameError: logOwner = LogOwner() ... # Prevent logfile from being erased on reload. This only works in cpython. try: logfile except NameError: logfile = NullFile() logerr = sys.stderr Okay, so we've got some code duplication here, and comment skew. I've fixed that up, and tidied up a few other bits of the code. And for the life of me I can't figure out why the log file should default (when no logging is enabled) to the NullFile rather than sys.stderr -- if you really want the NullFile, just pass it in. So I switched it to use sys.stderr. I then reran the 'admin/runtests' script and got ... some warnings and comments which were being supressed. /Users/dalke/cvses/Twisted/twisted/test/test_delay.py:24: DeprecationWarning: twisted.python.delay is deprecated. Please use reactor methods. from twisted.python import delay /Users/dalke/cvses/Twisted/twisted/protocols/dns.py:46: RuntimeWarning: PyCrypto not available - proceeding with non-cryptographically secure random source Enabling Multithreading. Resolver added ('64.105.156.138', 53) to server list Resolver added ('64.105.132.250', 53) to server list Also, I removed those os.linesep references, and the os module, and the needless string module. And turned 'file_protocol' into '_file_protocol' since it's for internal use only. Why's the Log pickleable? What happens if a cStringIO is passed in instead of a file? Or an outgoing network socket? Or a os.popen("lp")? It looks like Log.synchronized is old code that can be removed. Yes? Why does doc/examples/cursesclient.py set log.logfile directly? Shouldn't it call log.startLogging(log.NullFile()) ? It's the only other piece of code which references log.NullFile. Is 'discardLogs' the true inverse of startLogging? If so, it should add something to restore sys.stdout/sys.stderr if startLogging asked them to be changed. If not, where's the 'stopLogging' function? And 'discard' means to ignore logging? Or should it throw away the log files? The terminology is confusing. Anyway, the diff -u is attached for you all to review and correct. Oh, and with my changes, I now see the expected warning when I start up, and I see two twisted messages ... [Andrew-Dalkes-Computer:~/cvses/Twisted] dalke% python cansmi_rpc.py /usr/local/lib/python2.3/site-packages/openeye/oechem.py:5: RuntimeWarning: Python C API version mismatch for module _oechem: This Python has API version 1012, module _oechem has version 1011. import _oechem twisted.web.server.Site starting on 8080 Starting factory <twisted.web.server.Site instance at 0x640850> which tells me that those two messages ("Site starting" and "Starting factory") were being sent to never-never-land the whole time. Perhaps the latter shouldn't be present? Andrew dalke@dalkescientific.com