[Tutor] Do you use doctests to show API use even when you normally use unit tests?

[DL Neil]

On 4/05/20 1:08 AM, Oscar Benjamin wrote:
> On Sun, 26 Apr 2020 at 05:22, boB Stepp <robertvstepp at gmail.com> wrote:
>>
>> They discuss the
>> question of doctests versus unit tests, and, of course, heavily vote
>> for unit testing as the way to go.  However, they never stop inserting
>> doctests into their function and method docstrings.  I am assuming
>> that they consider it best practice as to how to document how to use a
>> function or method.
>>
>> My question is:  Is this what you would consider to be best
>> documentation practice?
> 
> Best documentation practice is to think carefully about what is the
> clearest way to explain something to users. Even a single very simple
> example can do a lot to clarify a description. Of course the doctest
> can only give examples so the description is always needed to explain
> things like the full scope of the function.
> 
> You shouldn't think of doctests as an alternative to unit tests. Unit
> tests are for testing the code whereas doctests are for testing the
> docs. As well as helping to clarify the usage of a function to end
> users doctests also remind authors of the code to keep the
> documentation up to date. In a larger project this means that if any
> contributor wants to change the behaviour of a function then they will
> have to update the doctests which should then trigger a review of the
> docstring as a whole (either by the author or by whoever reviews their
> work).

The word "users" is a little ambiguous - even though we (all) use it, 
all the time - the "user" of a library is likely to be a programmer, 
whereas the "user" of an application is likely a 'normal person'. 
(above, the distinction seems clear to me, but when it comes to 
documentation, both definitions apply - but to different documents, 
written at different levels of complexity and for different audiences)


This is good advice.

I often think the TDD cycle is slightly wrong (apologies to all who 
reacted to such heresy with a sharp intake of breath). Instead of the 
usually summarised cycle of: designing a test, then writing the code, 
then running (all) tests, and finally refactoring (either?both); I've 
thought it should be:

1 understand the purpose of the next step
2 in the code, add comment(s) which document 'the objective' AND 
anything necessary about 'the solution'
3 then write the test(s) - informed by the previous
4 write the code - equally so
etc

Will welcome critique...


Meantime, you may like to review Divio's "documentation system" 
(https://documentation.divio.com/). They identify four main types of 
documentation: Reference, Explanation, Tutorial, and How-To.

Which of these are relevant to this conversation?

I doubt that docstrings and supportive comments are the place for 
tutorials. Indeed most of the doctests mentioned earlier are at a level 
where 'tutorial' would be inappropriate - for example.

However, one of the irritating, almost arrogant, features of many F/LOSS 
projects (not specifically Python then) is the way some authors insist 
that the way to learn about their grand-creation, is to read the code 
(cf RTFM!). This is true, and doubly-so for security reasons, but... it 
is also a high 'barrier to entry'. Many a project would receive better 
take-up if it explained its API, at least with How-to examples. Initial 
application would be a lot faster (and motivational!), per @Alan's comment.

Par example: how easy did you find learning Python, compared with some 
other language? Did the documentation library support your efforts to a 
greater or lesser degree than equivalents?


So, my 2-cents: doctests are not so much useful for unit testing (I am 
using Pytest for that, at the moment), but for illustrating a routine's 
API and the variations, combinations, and permutations the code has been 
built to handle/reject. That they may also be used for 'unit testing' is 
somewhat, bonus-value!


Lastly, let me toss into the arena of your considerations, the idea that 
if all of the (functional) code was 'lost', would your testing suite 
(largely unit- and functional-/integration-tests) be sufficient to 
enable reconstruction?

In which case, would much of the 'added' documentation I described 
earlier (assuming your concurrence) be better placed in the tests rather 
than 'cluttering up' the functional-code modules?
-- 
Regards =dn