[Tutor] How do I test file operations (Such as opening, reading, writing, etc.)?

[Martin A. Brown]

Hello all,

[much snipped]

boB>> >> I don't want to mess with what will become the program's *real*
boB>> >> classifiers.txt (And other needed text files to come, that will
boB>> >> likewise be editable.), so how do I simulate these various needed file
boB>> >> operations in a way that tests the actual program code, but without
boB>> >> touching the actual data files?

Danny> As Alan says, you can also parameterize in a different way: by the
Danny> directory location where files are being read.  Then you can use a
Danny> temporary directory for your unit tests, and prepare the testing
Danny> environment that way.  If you take this approach, the tempfile module
Danny> can help with this.
Danny>
Danny>     https://docs.python.org/3.5/library/tempfile.html
Danny>     https://docs.python.org/3.5/library/tempfile.html#tempfile.TemporaryDirectory
Danny>

Alan>> > Danny has shown you one way using a mocked filesystem.
Alan>> > But for your case can't you just specify a file location
Alan>> > as an environment variable or argv? That way you get the
Alan>> > advantage of using real files, which can be an important
Alan>> > factor in timing issues, especially if you plan on having
Alan>> > any concurrency going on. And it's simple to do...

Matt>I would be tempted to generate a 'test data.CSV" file, and run the 
Matt>tests on that. It means that as you write the code, and find some 
Matt>edge cases, you can alter your code and add the edge cases to the 
Matt>test data. That way, the test data acts to cover the space of 
Matt>various oddities in your work.

I'll add one option to the mix, and summarize the other options I saw listed
earlier.

Option A (tempfile):

  Create a directory and copy your pristine tree into the directory
  and make the base directory for the important data files configurable.
  Also known as parameterization.

Option B (starting with Danny's sample code)

  Create the objects to emulate whatever filesystem behaviour you need.

Option C (use pyfakefs):

  Use pyfakefs, which acts like a filesystem for testing purposes.
  https://pypi.python.org/pypi/pyfakefs
  https://github.com/jmcgeheeiv/pyfakefs

Option D (use StringIO):

  If you are less concerned about filesystem interactions and really 
  just want an individual thingy that that behaves like a file, but 
  can be constructed in memory, use StringIO (or cStringIO).

Good luck!

-Martin

-- 
Martin A. Brown
http://linux-ip.net/