guidance sought: merging port related changes to Library modules
In preparing a set of patches intended to bring the OS/2 EMX port into CVS, I have a dilemma as to how best to integrate some changes to standard library modules. As background to this request I note that EMX and Cygwin have similar philosophies and attributes, being Posix/Unixish runtime environments on OSes with PC-DOS ancestry. Both rely on the GNU toolchain for software development. As a result of feedback on the previous set of patches, I am pruning cosmetic changes and attempting to minimise the footprint of the necessary changes. The particular changes I am looking for guidance on (or BDFL pronouncement on, as the case may be) involve os.py and the functionality in ntpath.py. The approach used in the port as released in binary form was to create a module called os2path.py (probably should really be called os2emxpath.py), which replicates the functionality of ntpath.py with OS2/EMX specific changes. Most of the changes have to do with using different path separator characters, with a few other changes reflecting slightly different behavour under EMX. EMX promotes the use of '/' as the path separator rather than '\', though it works with the latter. I don't know if Cygwin promotes the same convention. If I were to merge os2path.py into ntpath.py (which I incline towards instinctively) I believe that using references to os.sep and os.altsep rather than explicit '\\' and '/' strings would significantly reduce the extent of conditionalisation required, but in the process introduce significant source changes into ntpath.py (although the logical changes would be much less significant). If rationalising the use of separator characters (by moving away from hard-coded strings) in ntpath.py is unattractive, then I think I'd prefer to keep os2path.py (renamed to os2emxpath.py) as is, rather than revert to the DOS standard path separators. -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia
The particular changes I am looking for guidance on (or BDFL pronouncement on, as the case may be) involve os.py and the functionality in ntpath.py.
The approach used in the port as released in binary form was to create a module called os2path.py (probably should really be called os2emxpath.py), which replicates the functionality of ntpath.py with OS2/EMX specific changes.
Most of the changes have to do with using different path separator characters, with a few other changes reflecting slightly different behavour under EMX. EMX promotes the use of '/' as the path separator rather than '\', though it works with the latter. I don't know if Cygwin promotes the same convention.
If I were to merge os2path.py into ntpath.py (which I incline towards instinctively) I believe that using references to os.sep and os.altsep rather than explicit '\\' and '/' strings would significantly reduce the extent of conditionalisation required, but in the process introduce significant source changes into ntpath.py (although the logical changes would be much less significant).
If rationalising the use of separator characters (by moving away from hard-coded strings) in ntpath.py is unattractive, then I think I'd prefer to keep os2path.py (renamed to os2emxpath.py) as is, rather than revert to the DOS standard path separators.
The various modules ntpath, posixpath, macpath etc. are not just their to support their own platform on itself. They are also there to support foreign pathname twiddling. E.g. On Windows I might have a need to munge posix paths -- I can do that by explicitly importing posixpath. Likewise the reverse. So I think changing ntpath.py to use os.set etc. would be wrong, and creating a new file os2emxpath.py is the right thing to do -- despite the endless cloning of the same code. :-( (Maybe a different way to share more code between the XXXpath modules could be devised.) --Guido van Rossum (home page: http://www.python.org/~guido/)
If rationalising the use of separator characters (by moving away from hard-coded strings) in ntpath.py is unattractive, then I think I'd prefer to keep os2path.py (renamed to os2emxpath.py) as is, rather than revert to the DOS standard path separators.
I think replacing hard-coded separators by os.sep is a good thing to do. However, if you find that you cannot achieve re-use of ntpath for OS/2 by existing customization alone, please do not add conditional code into ntpath. It would be very confusing if, in ntpath.py, there is a test whether the system is OS/2. Regards, Martin
[Guido]
The various modules ntpath, posixpath, macpath etc. are not just their to support their own platform on itself. They are also there to support foreign pathname twiddling. E.g. On Windows I might have a need to munge posix paths -- I can do that by explicitly importing posixpath. Likewise the reverse.
Bingo.
So I think changing ntpath.py to use os.set etc. would be wrong, and creating a new file os2emxpath.py is the right thing to do -- despite the endless cloning of the same code. :-( (Maybe a different way to share more code between the XXXpath modules could be devised.)
Create _commonpath.py and put shared routines there. Then a blahpath.py can do from _commonpath import f, g, h to re-export them. An excellent candidate for inclusion would be expandvars(): the different routines for that now have radically different behaviors in endcases, it's impossible to say which are bugs or features, yet the routine *should* be wholly platform-independent (no, the Mac doesn't need its own version -- when an envar isn't found, the $envar token is retained literally). Having different versions of walk() is also silly, ditto isdir(), etc.
On Sat, 12 Jan 2002, Guido van Rossum wrote:
The various modules ntpath, posixpath, macpath etc. are not just their to support their own platform on itself. They are also there to support foreign pathname twiddling. E.g. On Windows I might have a need to munge posix paths -- I can do that by explicitly importing posixpath. Likewise the reverse.
So I think changing ntpath.py to use os.set etc. would be wrong, and creating a new file os2emxpath.py is the right thing to do -- despite the endless cloning of the same code. :-( (Maybe a different way to share more code between the XXXpath modules could be devised.)
I'd not considered the foreign path munging use, which I agree justifies retaining hardcoded path separators. I'll proceed on the basis of the os2emxpath.py approach. I'll pass for the time being on Tim's suggested rationalisation approach though... Thanks for the enlightment. -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au | Snail: PO Box 370 andymac@pcug.org.au | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia
Guido van Rossum writes:
The various modules ntpath, posixpath, macpath etc. are not just their to support their own platform on itself. They are also there to
Note that ntpath.abspath() relies on nt._getfullpathname(). It is not unreasonable for this particular function to require that it actually be running on NT, so I'm not going to suggest changing this. On the other hand, it means the portable portions of the module are (mostly) not tested when the regression test is run on a platform other than Windows; the ntpath.abspath() test raises an ImportError since ntpath.abspath() imports the "nt" module within the function, and the resulting ImportError causes the rest of the unit test to be skipped and regrtest.py reports that the test is skipped. I'd like to change the test so that the abspath() test is only run if the "nt" module is available: try: import nt except ImportError: pass else: tester('ntpath.abspath("C:\\")', "C:\\") Any objections? -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> PythonLabs at Zope Corporation
Fred writes:
Guido van Rossum writes:
The various modules ntpath, posixpath, macpath etc. are not just their to support their own platform on itself. They are also there to
Note that ntpath.abspath() relies on nt._getfullpathname(). It is not unreasonable for this particular function to require that it actually be running on NT, so I'm not going to suggest changing this. On the other hand, it means the portable portions of the module are (mostly) not tested when the regression test is run on a platform other than Windows; the ntpath.abspath() test raises an ImportError since ntpath.abspath() imports the "nt" module within the function, and the resulting ImportError causes the rest of the unit test to be skipped and regrtest.py reports that the test is skipped.
I'd like to change the test so that the abspath() test is only run if the "nt" module is available:
Sigh - this too would be my fault :( Before _getfullpathname() was added to the 'nt' module, there was an attempt to import 'win32api', and if OK, use the equivilent function from that. When I added the new function to 'nt', I removed that import check, in the belief it would now always succeed. This was obviously a bad call ;) (FYI, that was rev 1.35 of ntpath.py) A patch that reinstates the code would be: Index: ntpath.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/ntpath.py,v retrieving revision 1.44 diff -u -r1.44 ntpath.py --- ntpath.py 2001/11/05 21:25:02 1.44 +++ ntpath.py 2002/01/16 05:35:19 @@ -457,8 +457,18 @@ # Return an absolute path. def abspath(path): """Return the absolute version of a path""" - if path: # Empty path must return current working directory. + try: from nt import _getfullpathname + except ImportError: # Not running on Windows - mock up something sensible. + global abspath + def _abspath(path): + if not isabs(path): + path = join(os.getcwd(), path) + return normpath(path) + abspath = _abspath + return _abspath(path) + + if path: # Empty path must return current working directory. try: path = _getfullpathname(path) except WindowsError: This should also solve the test case problem. Thoughts? Mark.
Mark Hammond writes:
Before _getfullpathname() was added to the 'nt' module, there was an attempt to import 'win32api', and if OK, use the equivilent function from that. When I added the new function to 'nt', I removed that import check, in the belief it would now always succeed. This was obviously a bad call ;) (FYI, that was rev 1.35 of ntpath.py)
A patch that reinstates the code would be: ... This should also solve the test case problem.
I haven't tested this, but it looks OK to me. Feel free to check it in. Thanks! -Fred -- Fred L. Drake, Jr. <fdrake at acm.org> PythonLabs at Zope Corporation
participants (6)
-
Andrew MacIntyre
-
Fred L. Drake, Jr.
-
Guido van Rossum
-
Mark Hammond
-
Martin v. Loewis
-
Tim Peters