[ python-Bugs-542314 ] long file name support broken in windows

Mon Mar 26 14:36:22 CEST 2007

Bugs item #542314, was opened at 2002-04-11 06:23
Message generated for change (Comment added) made by titty
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=542314&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Windows
Group: Platform-specific
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Mark Hammond (mhammond)
Assigned to: Nobody/Anonymous (nobody)
Summary: long file name support broken in windows

Initial Comment:
>From c.l.py, thread &quot;&quot;

Peter D:
I'm using windows and trying to call os.path.getmtime
() after using os.path.walk... However, I'm choking 
with &quot;[Errno 38] Filename too long&quot; on paths with len 
&gt; ~260

Adding Martin's reply in a new comment (so it is not 
at the top of each and every mail I get on this bug :)

----------------------------------------------------------------------

Comment By: Ralf Schmitt (titty)
Date: 2007-03-26 14:36

Message:
Logged In: YES 
user_id=17929
Originator: NO

or if the c library is just insane:
from http://msdn2.microsoft.com/en-us/library/aa365247.aspx:

Maximum Path Length

In the Windows API, the maximum length for a path is MAX_PATH, which is
defined as 260 characters. A path is structured in the following order:
drive letter, colon, backslash, components separated by backslashes, and a
null-terminating character, for example, the maximum path on the D drive is
D:\<256 chars>NUL.

The Unicode versions of several functions permit a maximum path length of
approximately 32,000 characters composed of components up to 255 characters
in length. To specify that kind of path, use the "\\?\" prefix.

    Note  The maximum path of 32,000 characters is approximate, because
the "\\?\" prefix can be expanded to a longer string, and the expansion
applies to the total length.

For example, "\\?\D:\<path>". To specify such a UNC path, use the
"\\?\UNC\" prefix. For example, "\\?\UNC\<server>\<share>". These prefixes
are not used as part of the path itself. They indicate that the path should
be passed to the system with minimal modification, which means that you
cannot use forward slashes to represent path separators, or a period to
represent the current directory. Also, you cannot use the "\\?\" prefix
with a relative path. Relative paths are limited to MAX_PATH characters.

When using the API to create a directory, the specified path cannot be so
long that you cannot not append an 8.3 file name.

The shell and the file system may have different requirements. It is
possible to create a path with the API that the shell UI cannot handle.

-----

the explicit checks for pathnames longer than 260 characters seem to be
removed in python 2.5
and one can use the "magic prefix" to stat/create files with pathnames
longer than 260 characters.

----------------------------------------------------------------------

Comment By: Stuart Norton (stunorton)
Date: 2005-03-30 18:54

Message:
Logged In: YES 
user_id=1152606

I came across this suggestion while googling... and I would 
have expected it to work, but this code:

import os

os.chdir("\\\\ussvs-file02
\\radpubs\\wip\\zStaging\\translation\\D10
\\previous_test\\cumulative\\Common\\Reference\\API\\Borland
.Eco.Persistence.Configuration\\classes\\PersistenceMapper
DefinitionCollection\\Methods")
for filename in os.listdir("."):
    print filename
    infile = file(filename)

gives me 

C:\Documents and 
Settings\snorton\Desktop\h2build\buildtools>test.py
PersistenceMapperDefinitionCollection.AddRange.xml
PersistenceMapperDefinitionCollection.Assign.xml
PersistenceMapperDefinitionCollection.FindByName.xml
PersistenceMapperDefinitionCollection.NameExists.xml
PersistenceMapperDefinitionCollection.PersistenceMapperDefi
nitionCollection.xml
Traceback (most recent call last):
  File "C:\Documents and 
Settings\snorton\Desktop\h2build\buildtools\test.py", line 6, 
in ?
    infile = file(filename)
IOError: [Errno 2] No such file or 
directory: 'PersistenceMapperDefinitionCollection.Persistence
MapperDefinitionCollection.xml'

... funny that the file with the long path comes up in listdir() 
but isn't found with file()...

----------------------------------------------------------------------

Comment By: Mark Hammond (mhammond)
Date: 2002-04-11 06:26

Message:
Logged In: YES 
user_id=14198

Martin v. Loewis's reply on c.l.py:

Since you are asking for a work-around: cd into one of the 
more nested directories when the path gets longer 
(os.chdir), and use relative paths from then on.

If you want to study how to solve the problem: the relevant 
code snippet is in Modules/posixmodule.c

	/* the library call can blow up if the file name is 
too long! */
	if (pathlen &gt; MAX_PATH) {
		PyMem_Free(pathfree);
		errno = ENAMETOOLONG;
		return posix_error();
	}

There might be different ways to approach this:

- challenge the correctness of the comment:
   - try to establish why the author of that code
     thought that the C library could blow up
   - analyse whether these arguments are still true
     with the current compiler version
  -or-
   - prove the argument wrong by analysing the source 
     code of the C library
- then remove the constraint
-or-
- use different API to achieve the same effect without
  suffering from the constraint.

I'm serious about these suggestions: users would appreciate 
if this gets fixed somehow - apparently, the system allows 
longer file names, and apparently, the system itself can 
deal with that quite well. This
can be only true if the system either doesn't use its C 
library, or if the C library does not have this restriction.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=542314&group_id=5470