Handling test data that depends on temporal data (that might change while the test runs)

[Gabriel Genellina]

En Thu, 15 May 2008 14:21:34 -0300, Marcelo de Moraes Serpa  
<celoserpa at gmail.com> escribió:

> So, I have this particular method, generate_chat_dir_string, which should
> generate a string in the following format:
>  "md5hexstring-day-month-year-hour-minute"
> This string will be used to create a directory in the filesystem.

There is an issue here - how much behavior is considered in the public  
interfase, and how much is just an implementation detail. You only have to  
test the public interfase, what the method promises to do, NOT any  
internal implementation details.

I don't think that the md5 string be part of the interfase - probably it's  
enough to promise different directory names for different combinations of  
"user" and "atendente". A separate question: do we promise to generate the  
same directory name for the same set of user/atendente, or not? (assuming  
the time part is constant).
For the time part - is that format *required*, or just a means to obtain  
separate directory names, even for the same set of users, based on time?
Or perhaps we *only* want to guarantee different results for different  
function calls, and ALL that stuff involving users and time are just  
implementation details?
Depending on how you answer those questions, you write tests in a way or  
another way.

> I'm trying to adopt the TDD approach, so, I'm starting by testing all the
> methods of the Chat class, including this one. However, an issue just  
> popped
> up, this is the test code:
>
>
>    1.   def test_generate_chat_dir_string(self):
>    2.         import md5
>    3.         import random
>    4.         import datetime
>    5.         md5_handler = md5.new()
>    6.         users = [{"user":"pedro","atendente":False},{
>    "user":"joão","atendente":True}]
>    7.         #salt = random.random().to_s()
>    8.         #md5_handler.update(salt)
>    9.         now = datetime.datetime.now()
>    10.         ts = now.strftime("%d-%m-%Y-%H-%M")
>    11.         for user in users:
>    12.             md5_handler.update(user["user"])
>    13.
>    14.
>    15.         seed = md5_handler.hexdigest()
>    16.
>    17.         final_string = seed + "-" + ts
>    18.
>    19.         method_generated_string =  
> self.chat.generated_chat_dir_string
>    (users,seed)
>    20.
>    21.         self.assertEquals(final_string,method_generated_string)

Looks like you have just copied the generated_chat_dir_string  
implementation here - that's not the point of testing, at least if you  
relax the restrictions as I said above.

Supose the specification says, instead: the resulting directory must  
differ whether the set of users/atendente differs, or the request time is  
separated more than 1 minute apart.

Then the tests should cover:
- build a set of users and choose a point in time
- generate dir1 with that set of users and that time
- modify the set of users, generate dir2 and assert that it differs from  
dir1
   Repeat other modifications, including:
     one more user
     one less user
     same users, changing "atendente" for some
     only one user
     empty set of users (if it should raise an exception, check that it's  
actually raised)
     what happens when only the *order* of users is changed? (why is it a  
list instead of a set?)
     other conditions that might fail, perhaps based on your knowledge of  
the implementation (it's ok to test special cases that are special only  
because of how it is implemented, as far as you only test what is required  
by the specification). By example, suppose the md5 function is absolutely  
dumb and only accepts strings at most 32 characters long; test with a  
username longer than that.

- Then, increment the time less than a minute and verify you *can*  
generate dir3 (our specification doesn't *require* it to be the same, so  
don't check more than required)
- increment the time more than a minute, generate another dir and verify  
it's different.

Notice that, to be able to make all those tests involving the *same* time  
we should process them very quickly so the minute part doesn't change (but  
it even might fail when we're near the minute end). Worse, the last test  
woul require to *wait* more than one minute to see the change...
That's horrible - so we write the function in a way that *helps* testing.  
By example, adding another argument: the time to be used, which defaults  
to None meaning the current time.

def generated_chat_dir_string(self, users, curtime=None):
     if curtime is None: curtime = time.time()
     ...

(usually curtime is unused, but we do use it for testing)

I hope it's more clear now how to define your tests.

-- 
Gabriel Genellina