In an unprecedented move MacPython 2.2a3 is available only a scant 26 hours or so after the unix/windows distribution of 2.2a3.
MacPython 2.2a3 is available via http://www.cwi.nl/%7Ejack/macpython.html, as usual, and in hqx or macbinary form, as a full installer, an active installer and a source distribution (as usual:-).
Aside from the general new 2.2a3 features there are three specific changes in MacPython that are worth mentioning: - The structure of the MacOS toolbox modules has changed. All the modules have been put into a "Carbon" package (which, despite the name, runs fine in the classic PPC runtime model). There is a backwards compatibility folder on sys.path that will keep imports with the old names working (with an obnoxious warning). - Plugin modules are now in :Lib:lib-dynload in stead of in :Mac:PlugIns, to make the installed tree look more like the unix tree. - On input, unix line-endings are now acceptable for all text files. This is an experimental feature (awaiting a general solution, for which a PEP has been promised but not started yet, the giulty parties know who they are:-), and it can be turned off with a preference.
The downside of the quick release is that the installer has only been tested on MacOSX 10.0.4 and MacOS 9.1. Please report problems on older releases of MacOS asap. -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.cwi.nl/~jack | ++++ see http://www.xs4all.nl/%7Etank/ ++++
Jack Jansen wrote:
- On input, unix line-endings are now acceptable for all text files. This is an experimental feature (awaiting a general solution, for which a PEP has been promised but not started yet, the giulty parties know who they are:-), and it can be turned off with a preference.
I don't know if I qualify as one of the "guilty" parties, but I did volunteer to help with a PEP about this, and I'd still like to. I do have some ideas about what I'd like to see in that PEP.
The one thing I have done is write a prototype in pure Python for how I would like platform neutral text files to work. I've enclosed it with this message, and invite comments.
Has anyone started this PEP yet? if so, I'd like to help, if not, then the following is a very early draft of my thoughts. Note that I am writting this from memory, without going back to the archives to see what all the comments were at the time. I will do that before I call this a PEP.
Here are my quick thoughts:
This started (the recent thread, anyway) with the need for MacPython (with the introduction of OS-X) to be able to read both traditional mac style text files and unix style text files. An import-hook was suggested, but then it was brought up that a lot of python code can be read in other ways than an import, from execfile(), and a whole lot of others, so an imprt hook would not be enough. In general, the problem stems from the fact that while Python knows what system it is running on, a file that is being read may or may not be on that same system. This is most agregeuos with OS-X as you essentially have both Unix and MacOS running on the same machine at the same time, often sharing a file system. The issue also comes up with heterogeneous networks, where the file might reside on a server running on a different system than Python, and that file may be accessed by various systems. Some servers can do line feed translation on the fly, but this is not universal or foolproof.
In addition to Python code, many Python programs need to read and write text files that are not in a native format, and the format may not be known by the programmer when the code is writen.
My proposed solution to these problems is to have a new type of file: a "Universal" text file. This would be a text file that would do line-feed translation to the internal representation on the fly as the file was being read (like the current text file type), but it would translate any of the known text file formats automatically (\r\n, \r, \n Any others???). When the file was being written to, a single terminator would have to be specified, defaulting to the native one, or in the case of a file opened for appending, perhaps the one in the file when it is opened. The user could specify a non-native terminator when openign a file for writing.
The two big issues that came up in the discussion were backward compatability and performance:
1) The python open() function currently defaults to a text file type. However, on Posix systems, there is no difference between a text file and a binary file, so many programmers writing code that is designed to run only on such systems left the "b" flag off when opening files for binary reading and writing. If the behaviour of a file opened without the binary flag were to change, a lot of code would break.
2) In recent versions of Python, a lot of effort was put into improving performance of line oriented text file reading. These optimisations require the use of native line endings. In order to get similar performance with non-native endings, some portions of the C stdio library would have to be re-written. This is a major undertaking, and no one has stepped up to volunteer.
The proposed solution to both of these problems is to introduce a new flag to the open() function: "t". If the "t" flag is present, the function returns a Universal Text File, rather than a standard text file. As this is a new flag, no old code should be broken. The default would return a standard text file with the current behaviour. This would allow the implimentation to be written in a way that was robust, but perhaps not have optimum performance. If performance were critical, a programmer could always use the old style text file. If, at some point, code is written that allows the performance of Universal Text Files to approach that of standard text files, perhaps the two could be merged. It is unfortunate that the default would be the performance-optimised but less generally useful case, but that is a reasonable price to be paid for backward compatability. Perhaps the default could be changed at some point in the future when other incompatabilities are introduced (Python 3?)
In the case of Python code being read, performance of the file read is unlikely to be critical to the performance of the application as a whole.
Issues / questions:
Some systems, (VMS ?) store text files in the file system as a series of lines, rather than just a string of bytes like most common systems today. It would take a little more code to accomidate this, but it could be done.
Should a file being read be required to have a single line termination type, or could they be mixed and matched? The prototype code allows mix and match, but I'm not married to that idea. If it requires a single terminator, then some performance could be gained by checking the terminator type when opening the file, and using the existing native text file code when it is a native file.
I'd love to hear all your feedback on this write-up, as well as my code. Please either CC me or the MacPython list, as I'm not subscribed to python-dev