Mailman 3 October 2007 - Python-Dev

OpenSSL httplib bug
by Richie Ward Oct. 1, 2007

Oct. 1, 2007

I was using httplib to power my xml rpc script. I had problems when I wanted to use SSL and I got this error: File "/usr/lib/python2.5/httplib.py", line 1109, in recv return self._ssl.read(len) socket.sslerror: (8, 'EOF occurred in violation of protocol') I figured out this was because of poor error handling in python. May I suggest this as a fix to this bug: $ diff /usr/lib/python2.5/httplib.py /usr/lib/python2.5/httplib.py~ 1109,1112c1109 < try: < return self._ssl.read(len) < except socket.sslerror: < return --- > return self._ssl.read(len) Just a note. I am by no means a python expert, just good enough to get my work done. I use Ubuntu gutsy. -- Thanks, Richie Ward

3 2

building with -Wwrite-strings
by Lisandro Dalcin Oct. 1, 2007

Oct. 1, 2007

I'm trying to build Python (2.6) with GCC the option -Wwrite-strings. 1 - Is there any interest on this? 2 - What should I do for the very common (taken from int_new): static char *kwlist[] = {"x", "base", 0}; I was able to remove all the warning in Objects/*, except those related to (2). -- Lisandro Dalcín --------------- Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC) Instituto de Desarrollo Tecnológico para la Industria Química (INTEC) Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) PTLC - Güemes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594

3 3

Re: [Python-Dev] [python] Re: New lines, carriage returns, and Windows
by Nick Maclaren Oct. 1, 2007

Oct. 1, 2007

"Paul Moore" <p.f.moore(a)gmail.com> wrote: > > > So, damn the outside system, EXACTLY what does Python mean by > > such characters, and EXACTLY what uses of them are discouraged > > as having unspecified meanings? If we could get an answer to > > that precisely enough to write a parse tree with all terminals > > explicit, this problem would go away. > > Python, the language, means nothing by the characters. They are bytes > with defined values in a byte string (in 2.x, in 3.0 they are Unicode > characters, but otherwise no difference). The *language* places no > interpretation on them. Actually, it's not that simple, because of the "universal newline" rule and the fact that both Unix/C ASCII and Unicode DO provide meanings for their characters, but let that pass. Your statement is not far off the situation. > Certain library functions place an interpretation on the byte values, > but you need to read the function definition for that. And (a) they > may not all be consistent, and (b) they may say "follows platform > behaviour", but that's the way it is, so you have to live with it. And that is why there will continue to be confusion and inconsistency, and why there will be similar threads to this for the foreseeable future. If you regard continuing problems of this sort as acceptable, then fine, but I am pointing out that they are fairly easy to avoid. But only by specifying a precise Python model. Incidentally, the response (b) you give is a common one, but isn't usually correct when it is given. It is, after all, the cause of the problem that started this thread. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

New lines, carriage returns, and Windows
by Dino Viehland Oct. 1, 2007

Oct. 1, 2007

We ran into an interesting user-reported issue w/ IronPython and the way Python writes to files and I thought I'd get python-dev's opinion. When writing a string in text mode that contains \r\n we both write \r\r\n because the default write mode is to replace \n with \r\n. This works great as long as you stay within an entirely Python world. Because Python uses \n for everything internally you'll never end up writing out a \r\n that gets transformed into a \r\r\n. But when interoperating with other native code (or .NET code in our case) it's fairly easy to be exposed to a string which contains \r\n. Ultimately we see odd behavior when round tripping the contents of a multi-line text box through a file. So today users have to be aware of the fact that Python internally always uses \n. They also need to be aware of any APIs that they call that might return a string with an embedded \r\n inside of them and transform the string back into the Python version. It could be argued that there's little value in doing the simple transformation from \r\n -> \r\r\n. Ultimately that creates a file that has line endings which aren't good on any platform. On the other hand it could also be argued that Python defines new-lines as \n and there should be no deviation from that. And doing so could be considered a slippery slope, first file deals with it, and next the standard libraries, etc... Finally this might break some apps and if we changed IronPython to behave differently we could introduce incompatibilities which we don't want. So I'm curious: Is there a reason this behavior is useful that I'm missing? Would there be a possibility (or objections to) making \r\n be untransformed in the Py3k timeframe? Or should we just tell our users to open files in binary mode? :)

7 16

Re: [Python-Dev] New lines, carriage returns, and Windows
by Nick Maclaren Oct. 1, 2007

Oct. 1, 2007

Greg Ewing <greg.ewing(a)canterbury.ac.nz> wrote: > > I don't see how this is different from Unix/C "\n" being > an atomic newline character. Have you used systems with the I/O models I referred to (or ones with newlines being out-of-bound data)? > If you're saying that BCPL is better because it defines > standard semantics for more control characters than just > "\n", that may be true, but C is doing about the best it > can with "\n" as far as I can see, given all the crazy > things that different OSes want to do with line endings. I am afraid that you are wrong - see my other posting for how to do it better. Look, I have implemented both of those two models on systems that are FAR more different than most people can imagine. Both work, and neither causes confusion. The C/Unix/Python one does. > In any case, the problem which started all this isn't > really an I/O problem at all, it's a mismatch between > the world of Python strings which use "\n" and .NET > library code expecting strings which use "\r\n". That's an I/O problem :-) > The correct thing to do with that is to translate whenever > a string crosses a boundary between Python code and > .NET code. This is something that ought to be done > automatically by the Python/.NET interfacing machinery, > maybe by having a different type for .NET strings. Agreed. But the REASON it causes trouble is the inconsistency in the basic C/Unix/Python text I/O model. Let's consider just \f, \r and \n, and a few questions: Exactly what does a free-standing \f mean? Does \n\f\n mean starting at the top of a page or one line down? How do \r and \f interact with line-buffering? Think about MacOS here. I could go on, but those are enough to indicate that the problem is insoluble. The answer "Undefined but not even explicitly discouraged" is a recipe for confusion. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

2 1

Re: [Python-Dev] [python] Re: New lines, carriage returns, and Windows
by Nick Maclaren Oct. 1, 2007

Oct. 1, 2007

Greg Ewing <greg.ewing(a)canterbury.ac.nz> wrote: > > > Grrk. That's the problem. You don't get back what you have written > > You do as long as you *don't* use universal newlines mode > for reading. This is the best that can be done, because > universal newlines are inherently ambiguous. I don't know PRECISELY what you mean by "universal newlines mode", and this issue is all about the details, so any response would merely enhance the confusion. > If you want universal newlines, you just have to accept > that you can't also have \r characters meaning something > other than newlines in your files. This is true regardless > of what programming language or I/O model is being used. No, that is not true, and I have used more than one model where it wasn't. Let's stick to models where newlines are special characters - I prefer the ones where they are not, but that is by the way. Model 1: certain characters can be used only in combination. E.g. \f must occur immediately before (or after) a \n, which it modifies. r is either a newline-with-overprint or must be associated with a \n. In both cases, only ONE of the alternatives is permitted in the chosen model - the other use then becomes an error (and raises an exception). Model 2: (BCPL) there are a variety of newline characters, \n for plain newline, \f for newline-with-form-feed and \r for newline- with-overprint. ALL cause a newline, with the associated property. Note that the above is what the program sees - what is written to the outside world and how input is read is another matter. But I can assure you, from my own and many other people's experience, that neither of the above models cause the confusion being shown by the postings in this thread. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

2 1