[Tutor] Writing unit tests that involve email

[Arnel Legaspi]

Steven D'Aprano <steve <at> pearwood.info> writes:
>
> On 14/07/13 10:32, Arnel Legaspi wrote:
> [...]
> > What I'm having difficulty is getting the unit tests to properly run the
> > script being tested and send the email, get the proper Maildir message,
> > and be able to read it via the unit test script. Right now, even when
> > the SMTP server is not running, all the tests for checking the sent
> > email pass, which is not what I intended.
> >
> > I know the SMTP server works, because if I send some email using another
> > Python or Powershell script, the script logs on the screen that it did
> > receive the email.
> >
> > How should I approach this?
> >
> > Here is the unit test script I have (some lines may get wrapped):
> 
> Firstly, start with the obvious: are you sure that the unittests are being
> run? If I have counted correctly, you only have seven test cases at the
> moment, so your unittest output should show seven dots and then print
> something like:
> 
> Ran 7 tests in 0.01s
> 
> OK
> 
> Can you confirm that this is the case? If not, then your problem is that 
the
> tests aren't running at all.

Yes, they do. All the tests pass, but when I look inside the Maildir
directories (and the dsmtpd log on the console), no email appears to 
get sent.

I was expecting something along the lines of (see the 3rd output line):

$ dsmtpd -d mail_dir -p 8000
2013-07-17 11:07:53,029 INFO: Starting dsmtpd 0.2.2 at 127.0.0.1:8000
2013-07-17 11:07:53,053 INFO: Store the incoming emails into mail_dir
2013-07-17 11:13:18,618 INFO: 127.0.0.1:38231: user at wherever.com -> 
test at email.net [Email Subject]

If I run this test script, I should be seeing 7 lines similar to it.

> [...]
> >       parser.set('smtp', 'password', ***redacted***)
> 
> I hope this isn't a real password to a real mail server visible from the
> internet, because you've now made it visible to everyone in the world.

Nope, it's not. It's really just for the unit test. (I also hope no one uses
"user at wherever.com" but it shouldn't be likely.)

> > class TestReadingSendingPostFile(unittest.TestCase):
> >
> >       def setUp(self):
> >           """ Prepare a post file for testing. """
> >           create_configfile()
> >           with open('post.md', 'wb') as postfile:
> >               postfile.write(MKDOWN_TEXT)
> >           self.mbox = mailbox.Maildir('mail_dir')
> 
> Personally, I don't use the setUp and testDown methods, but most of my 
code
> avoids storing state so they are rarely relevant to me. But in any case, 
the
> setUp method is called before *every* test method. If you want it to be
> called only once, you should put it inside the class __init__ method.
> (Don't forget to call the TestCase __init__ as well.)

I needed the setUp / tearDown methods because the script I wrote (on the
Bitbucket repo) reads email sending/receiving information from a config 
file.
Putting something in class __init__ method is new to me.
 
> I see all of your test methods include clear_args() at the start, so you
> should put that inside the setUp method. It would help if you tell us
> which test method you are expecting to fail. My guess is that it is this 
one:
> 
> >       def test_sendpost(self):
> >           clear_args()
> >           add_testfiles()
> >           for message in self.mbox:
> >               self.assertEqual(message['subject'], 'post')
> >               self.assertEqual(message.get_payload(), HTML_TXT)
> 
> I don't believe this actually tries to send any email, and as far as I can
> see you never actually clear the mailbox, so all you're doing is looking 
at
> old emails that happened to exist.
>
> If the mbox happens to be empty, then neither of the assertEqual calls 
will
> happen, and the test will just pass. Add this to the test:
> 
>            if not self.mbox:
>                self.fail("No messages were sent")
>
> or better still, add a separate test to check that email is sent, without
> caring about the content of the email. That way, if you get a failure, you
> can immediately see whether the failure is "email wasn't sent", or "email 
was
> sent, but contains the wrong stuff".

I did put in something like that before, which is how I confirmed the test 
was
not sending the emails like I wanted it to do.
 
> I suggest you factor out any tests of sending mail into a separate class.
> Give this class a setUp() that creates a mailbox, and a tearDown that 
moves
> everything from that mailbox into another one. That means you have a nice
> clean mailbox at the start of each test, but you can still inspect the 
emails
> by eye if you want. It also means that you can separate tests with an
> external dependency (the mail server) from those without.

All right, but the trouble I have is on making the unit tests run such that 
it
will force the script I'm testing to send the email. If I just use the 
script
on my own, it does send the emails, no problem. With the unit tests I've
written, it's not doing so.

Sorry if I come off a little dense, but I don't think I misread what you 
were
trying to say here.

Thanks,
Arnel