Mailman 3 October 2007 - Python-Dev

Returning exception stack trace in SimpleXMLRPCServer response
by Raghuram Devarakonda Oct. 4, 2007

Oct. 4, 2007

Hi, When an exception occurs in a function, SimpleXMLRPCServer currently sends exception type and value back to the client (in <faultString>) . So the client sees some thing like: "'<type \'exceptions.ValueError\'>:invalid literal for int() with base 10" This string is constructed in _marshaled_dispatch() as ""%s:%s" % (exc_type, exc_value)". How about adding the stack traceback to this string? I found it very useful to have the server side traceback in case of failures. I will come up with a patch if there is any interest. Some thing similar has been done in case of internal server errors (in r57158). Thanks, Raghu

1 0

Backporting Decimal
by Facundo Batista Oct. 2, 2007

Oct. 2, 2007

People: I don't decide myself what to do in this case. The Decimal module appeared in 2.4, and received just slight modifications for 2.5. Since it appeared, a "just download and use it" version was available for Python 2.3 users. But for 2.6, it was fully renewed. Not only was updated to the last spec from Cawlishaw, but also a lot of (very slightly, arithmetic corner cases) bugs were fixed. Also, it received a lot of code simplifications and speedups. All this is, in most part, because Mark Dickinson was involved here, and he did (and is doing) a great work. When 2.6 is out, I'll update that downloadable for Py2.3. But what to do with Py2.5? Decimal is a pretty stand alone module, and I'm absolutely sure that just backporting the whole module and its testcases will fix a lot of problems, and Py2.5 users will have new functionality, but is this ok? (of course, I can not be 100% sure that any bugs were introduced in these modifications). Recommendations? Thank you very much! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/

4 6

Re: [Python-Dev] [python] Re: New lines, carriage returns, and Windows
by Nick Maclaren Oct. 2, 2007

Oct. 2, 2007

"Guido van Rossum" <guido(a)python.org> wrote: > > Does anyone else have the feeling that discussions with Mr. MacLaren > don't usually bear any fruit? Yes. I do. My ability to predict the (technical) future is good; my ability to persuade people of it is almost non-existent. However, when an almost identical thread to this one occurs in a decade's time, I shall be safely retired. And, assuming no changes to the basic models, when one occurs in twenty years' time, I shall be forgotten. Such is life. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

Re: [Python-Dev] [python] Re: New lines, carriage returns, and Windows
by Nick Maclaren Oct. 2, 2007

Oct. 2, 2007

Greg Ewing <greg.ewing(a)canterbury.ac.nz> wrote: > > > I don't know PRECISELY what you mean by "universal newlines mode" > > I mean precisely what Python means by the term: any of > "\r", "\n" or "\r\n" represent a newline, and no distinction > is made between them. Excellent. While this over-simplifies the issue, let's stick to the over-simplified form, as we may be able to get somewhere. The question is independent of what the outside system believes a text file should look like, and is solely what Python believes a sequence of characters should mean. For example, does 'A\r\nB' mean that B is separated from A by one newline or two? The point is that, once we know that, we can design a translator to and from Python's conventions to any reasonable system (and, as I say, I have done it many times). But, if Python's own interpretation is ambiguous, it is a sure recipe for different translators being incompatible, even on the same system. Which is what has happened here. So, damn the outside system, EXACTLY what does Python mean by such characters, and EXACTLY what uses of them are discouraged as having unspecified meanings? If we could get an answer to that precisely enough to write a parse tree with all terminals explicit, this problem would go away. And that is all that I say can or should be done. The details of how to write the translators to other file systems are then a separate matter. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

5 4

Re: [Python-Dev] New lines, carriage returns, and Windows
by Nick Maclaren Oct. 2, 2007

Oct. 2, 2007

"Paul Moore" <p.f.moore(a)gmail.com> wrote: > > OK, so far so good - although I'm not *quite* sure there's a > self-consistent definition of "code that only uses \n". I'll assume > you mean code that has a concept of lines, that lines never contain > anything other than text (specifically, neither \r or \n can appear in > a line, I'll punt on whether other weird stuff like form feed are > legal), and that whenever your code needs to write data to a file, it > writes lines with \n alone between them. I won't. There are a few of us still left who know how this started, and here is a simplified description. Unix was a computer scientist's workbench, and made no attempt to be general. In particular, its text datastream model was appropriate for the imnportant devices of the day - teletypes and similar. So far, so good. But what was forgotten later is that the model does NOT extend to other systems and, in particular, made no sense on the record-oriented models generally used by mainframes (see Fortran for an example). When C was standardised, this was fudged. I tried to get it improved, but it is one of the many things I failed to do. The handling of ALL of the control characters in text I/O is non-portable (even \t, despite what the satndard says), and you have to follow the system's constraints if things are to work. Unfortunately, the kludging that the compiler does to map C to the operating system confuses things still further - though it is essential. Now, BCPL was an ancestor of C, but always was a more portable language (i.e. it didn't start with a specific operating system in mind), and used/uses a rather better model. In this, line separators are atomic - e.g. '\f' is newline-with-form-feed and '\r' is "newline-with-overprinting". Now, THAT model is more generic. Not fully generic, of course, but it would cater for all of Unix, CPM and its derivatives (yes, Microsoft), MacOS and most mainframes (with some reservations). So, until and unless Python chooses to define its own I/O model, these problems will continue to arise. Whether this one is a simple bug or an avoidable feature, I can't say without looking harder, but bugs are often caused by attempting to implement impossible or confusing specifications. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

10 21

OpenSSL httplib bug
by Richie Ward Oct. 1, 2007

Oct. 1, 2007

I was using httplib to power my xml rpc script. I had problems when I wanted to use SSL and I got this error: File "/usr/lib/python2.5/httplib.py", line 1109, in recv return self._ssl.read(len) socket.sslerror: (8, 'EOF occurred in violation of protocol') I figured out this was because of poor error handling in python. May I suggest this as a fix to this bug: $ diff /usr/lib/python2.5/httplib.py /usr/lib/python2.5/httplib.py~ 1109,1112c1109 < try: < return self._ssl.read(len) < except socket.sslerror: < return --- > return self._ssl.read(len) Just a note. I am by no means a python expert, just good enough to get my work done. I use Ubuntu gutsy. -- Thanks, Richie Ward

3 2

building with -Wwrite-strings
by Lisandro Dalcin Oct. 1, 2007

Oct. 1, 2007

I'm trying to build Python (2.6) with GCC the option -Wwrite-strings. 1 - Is there any interest on this? 2 - What should I do for the very common (taken from int_new): static char *kwlist[] = {"x", "base", 0}; I was able to remove all the warning in Objects/*, except those related to (2). -- Lisandro Dalcín --------------- Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC) Instituto de Desarrollo Tecnológico para la Industria Química (INTEC) Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) PTLC - Güemes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594

3 3

Re: [Python-Dev] [python] Re: New lines, carriage returns, and Windows
by Nick Maclaren Oct. 1, 2007

Oct. 1, 2007

"Paul Moore" <p.f.moore(a)gmail.com> wrote: > > > So, damn the outside system, EXACTLY what does Python mean by > > such characters, and EXACTLY what uses of them are discouraged > > as having unspecified meanings? If we could get an answer to > > that precisely enough to write a parse tree with all terminals > > explicit, this problem would go away. > > Python, the language, means nothing by the characters. They are bytes > with defined values in a byte string (in 2.x, in 3.0 they are Unicode > characters, but otherwise no difference). The *language* places no > interpretation on them. Actually, it's not that simple, because of the "universal newline" rule and the fact that both Unix/C ASCII and Unicode DO provide meanings for their characters, but let that pass. Your statement is not far off the situation. > Certain library functions place an interpretation on the byte values, > but you need to read the function definition for that. And (a) they > may not all be consistent, and (b) they may say "follows platform > behaviour", but that's the way it is, so you have to live with it. And that is why there will continue to be confusion and inconsistency, and why there will be similar threads to this for the foreseeable future. If you regard continuing problems of this sort as acceptable, then fine, but I am pointing out that they are fairly easy to avoid. But only by specifying a precise Python model. Incidentally, the response (b) you give is a common one, but isn't usually correct when it is given. It is, after all, the cause of the problem that started this thread. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0

New lines, carriage returns, and Windows
by Dino Viehland Oct. 1, 2007

Oct. 1, 2007

We ran into an interesting user-reported issue w/ IronPython and the way Python writes to files and I thought I'd get python-dev's opinion. When writing a string in text mode that contains \r\n we both write \r\r\n because the default write mode is to replace \n with \r\n. This works great as long as you stay within an entirely Python world. Because Python uses \n for everything internally you'll never end up writing out a \r\n that gets transformed into a \r\r\n. But when interoperating with other native code (or .NET code in our case) it's fairly easy to be exposed to a string which contains \r\n. Ultimately we see odd behavior when round tripping the contents of a multi-line text box through a file. So today users have to be aware of the fact that Python internally always uses \n. They also need to be aware of any APIs that they call that might return a string with an embedded \r\n inside of them and transform the string back into the Python version. It could be argued that there's little value in doing the simple transformation from \r\n -> \r\r\n. Ultimately that creates a file that has line endings which aren't good on any platform. On the other hand it could also be argued that Python defines new-lines as \n and there should be no deviation from that. And doing so could be considered a slippery slope, first file deals with it, and next the standard libraries, etc... Finally this might break some apps and if we changed IronPython to behave differently we could introduce incompatibilities which we don't want. So I'm curious: Is there a reason this behavior is useful that I'm missing? Would there be a possibility (or objections to) making \r\n be untransformed in the Py3k timeframe? Or should we just tell our users to open files in binary mode? :)

7 16

Re: [Python-Dev] New lines, carriage returns, and Windows
by Nick Maclaren Oct. 1, 2007

Oct. 1, 2007

Greg Ewing <greg.ewing(a)canterbury.ac.nz> wrote: > > I don't see how this is different from Unix/C "\n" being > an atomic newline character. Have you used systems with the I/O models I referred to (or ones with newlines being out-of-bound data)? > If you're saying that BCPL is better because it defines > standard semantics for more control characters than just > "\n", that may be true, but C is doing about the best it > can with "\n" as far as I can see, given all the crazy > things that different OSes want to do with line endings. I am afraid that you are wrong - see my other posting for how to do it better. Look, I have implemented both of those two models on systems that are FAR more different than most people can imagine. Both work, and neither causes confusion. The C/Unix/Python one does. > In any case, the problem which started all this isn't > really an I/O problem at all, it's a mismatch between > the world of Python strings which use "\n" and .NET > library code expecting strings which use "\r\n". That's an I/O problem :-) > The correct thing to do with that is to translate whenever > a string crosses a boundary between Python code and > .NET code. This is something that ought to be done > automatically by the Python/.NET interfacing machinery, > maybe by having a different type for .NET strings. Agreed. But the REASON it causes trouble is the inconsistency in the basic C/Unix/Python text I/O model. Let's consider just \f, \r and \n, and a few questions: Exactly what does a free-standing \f mean? Does \n\f\n mean starting at the top of a page or one line down? How do \r and \f interact with line-buffering? Think about MacOS here. I could go on, but those are enough to indicate that the problem is insoluble. The answer "Undefined but not even explicitly discouraged" is a recipe for confusion. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

2 1

Re: [Python-Dev] [python] Re: New lines, carriage returns, and Windows
by Nick Maclaren Oct. 1, 2007

Oct. 1, 2007

Greg Ewing <greg.ewing(a)canterbury.ac.nz> wrote: > > > Grrk. That's the problem. You don't get back what you have written > > You do as long as you *don't* use universal newlines mode > for reading. This is the best that can be done, because > universal newlines are inherently ambiguous. I don't know PRECISELY what you mean by "universal newlines mode", and this issue is all about the details, so any response would merely enhance the confusion. > If you want universal newlines, you just have to accept > that you can't also have \r characters meaning something > other than newlines in your files. This is true regardless > of what programming language or I/O model is being used. No, that is not true, and I have used more than one model where it wasn't. Let's stick to models where newlines are special characters - I prefer the ones where they are not, but that is by the way. Model 1: certain characters can be used only in combination. E.g. \f must occur immediately before (or after) a \n, which it modifies. r is either a newline-with-overprint or must be associated with a \n. In both cases, only ONE of the alternatives is permitted in the chosen model - the other use then becomes an error (and raises an exception). Model 2: (BCPL) there are a variety of newline characters, \n for plain newline, \f for newline-with-form-feed and \r for newline- with-overprint. ALL cause a newline, with the associated property. Note that the above is what the program sees - what is written to the outside world and how input is read is another matter. But I can assure you, from my own and many other people's experience, that neither of the above models cause the confusion being shown by the postings in this thread. Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

2 1