[Tutor] Why does os.path.realpath('test_main.py') give different results for unittest than for testing statement in interpreter?

[Steven D'Aprano]

On Sun, Jan 07, 2018 at 03:07:26AM -0600, boB Stepp wrote:

> After some searching I have yet to locate a definition of "canonical
> path" that makes sense to me.  The dictionary definition of canonical
> does not seem to be very helpful in understanding this.

Probably the best definition would be "the simplest authoritative form 
of the path".


> What is the methodology that os.path.realpath(path) is
> actually following to yield a particular path name?

Trust the Source, Luke :-)

The Python 3.5 source code for os.path.realpath under Windows looks like 
this:

# realpath is a no-op on systems without islink support
realpath = abspath

The comment is misleading[1] for two reasons:

- it isn't a no-op, since abspath() actually does something; 
  what they mean is that on Windows realpath() doesn't do
  anything beyond what abspath() does;

  (on Linux and Mac, it also checks for symbolic links, and
  converts them to their actual paths)

- and technically speaking, some Windows file systems *do*
  support symbolic links, although few people know this or
  use them.

https://en.wikipedia.org/wiki/NTFS_symbolic_link


Nevertheless, we can say that under Windows realpath() does nothing more 
than abspath() (at least up to Python 3.5). So what does abspath() do?

The source for abspath() is a more little complicated, because it has 
two different implementations depending on whether you are *actually* 
running Windows or not. But basically, abspath() does this:

- if the path is a relative path (does not start with C:\ or similar)
  then prepend the current directory to the path;

- then normalise the path.

What does it mean to normalise the path?

It converts it to the simplest form, eliminating redundant slashes and 
dots. Because this is Windows, there are a bunch of special cases for 
device names and literal paths (whatever they are!) that look like this:

    \\.\ -> device names
     \\?\ -> literal paths


but for the standard case of ordinary file names, normpath() on Windows 
will convert forward slashes / to backslashes \ and eliminate duplicated 
slashes and dots in the path. Remember that . means "this directory" and 
.. means "one directory up", normpath() will normalise any of these:

    A//B
    A/./B
    A/foo/../B

to A\B. Since the Windows version of realpath() doesn't care about 
symbolic links, it doesn't need to care whether any of the directories 
and files actually exist or not. All it needs to do is turn a relative 
path into an absolute path, then normalise the path. And none of that 
depends on the path actually existing.


> What I *really* want to do is locate and open a file "test_data.dat"
> that is in the same directory as the file with the tests,
> "tests/test_main.py".

If you can assume that tests/test_main.py is always in the current 
directory, then you can just write:

PATH = 'tests/test_main.py'

and use that. open(PATH) will happily work on relative paths as well as 
absolute.

Reminder: absolute paths start with a drive:

    A:\foo\bar

relative paths don't, and so they implicitly start in the current 
directory.

*But* that assumes that you have started running the tests from inside 
the directory which includes tests/test_main.py. That might not be the 
case. In order to solve this problem properly, I think we need to know 
how your project is laid out, including the tests.

But my guess is that the solution is to extract the path of the running 
module, and append tests/test_main.py to that path.

If this isn't clear, ask and I'll explain in more detail. (But later, 
I'm about to move off the computer for a few hours.)


> Anyway, based on that
> earlier related question and your answer tonight I have modified my
> test class to the following:
[...]

I'll read it later.




[1] "At Resolver we've found it useful to short-circuit any doubt and 
just refer to comments in code as 'lies'. "
--Michael Foord paraphrases Christian Muirhead on python-dev, 2009-03-22


-- 
Steve