Mailman 3 May 2000 - Python-Dev

issues with int/long on 64bit platforms - eg stringobject (PR#306)
by Guido van Rossum May 5, 2000

May 5, 2000

The email below is a serious bug report. A quick analysis shows that UserString.count() calls the count() method on a string object, which calls PyArg_ParseTuple() with the format string "O|ii". The 'i' format code truncates integers. It probably should raise an overflow exception instead. But that would still cause the test to fail -- just in a different way (more explicit). Then the string methods should be fixed to use long ints instead -- and then something else would probably break... … [View More] --Guido van Rossum (home page: http://www.python.org/~guido/) ------- Forwarded Message Date: Mon, 24 Apr 2000 19:26:27 -0400 From: mark.favas(a)per.dem.csiro.au To: python-bugs-list(a)python.org cc: bugs-py(a)python.org Subject: [Python-bugs-list] 1.6a2 issues with int/long on 64bit platforms - eg stringobject (PR#306) Full_Name: Mark Favas Version: 1.6a2 CVS of 25 April OS: DEC Alpha, Tru64 Unix 4.0F Submission from: wa107.dialup.csiro.au (130.116.4.107) There seems to be issues (and perhaps lurking cans of worms) on 64-bit platforms where sizeof(long) != sizeof(int). For example, the CVS version of 1.6a2 of 25 April fails the UserString regression test. The tests fail as follows (verbose set to 1): abcabcabc.count(('abc',)) no 'abcabcabc' <method UserString.count of UserString instance at 140217e40> 3 <> 2 abcabcabc.count(('abc', 1)) no 'abcabcabc' <method UserString.count of UserString instance at 140216580> 2 <> 1 abcdefghiabc.find(('abc', 1)) no 'abcdefghiabc' <method UserString.find of UserString instance at 140216730> 9 < > - -1 abcdefghiabc.rfind(('abc',)) no 'abcdefghiabc' <method UserString.rfind of UserString instance at 140216580> 9 <> 0 abcabcabc.rindex(('abc',)) no 'abcabcabc' <method UserString.rindex of UserString instance at 140216dc0> 6 <> 3 abcabcabc.rindex(('abc', 1)) no 'abcabcabc' <method UserString.rindex of UserString instance at 140216730> 6 <> 3 These tests are failing because the calls from the UserString methods to the underlying string methods are setting the default value of the end-of-string parameter to sys.maxint, which is defined as LONG_MAX (9223372036854775807), whereas the string methods in stringobject.c are using ints and expecting them to be no larger than INT_MAX (2147483647). Thus the end-of-string parameter becomes -1 in the default case. The size of an int on my platform is 4, and the size of a long is 8, so the "natural size of a Python integer" should be 8, by my understanding. The obvious fix is to change stringobject.c to use longs, rather than ints, but the problem might be more widespread than that. INT_MAX is used in unicodeobject.c, pypcre.c, _sre.c, stropmodule.c, and ceval.c as well as stringobject.c. Some of these look as though LONG_MAX should have been used (variables compared to INT_MAX are longs, but I am not confident enough to submit patches for them... Mark _______________________________________________ Python-bugs-list maillist - Python-bugs-list(a)python.org http://www.python.org/mailman/listinfo/python-bugs-list ------- End of Forwarded Message [View Less]

5 8

Re: [I18n-sig] Unicode strings: an alternative
by Florian Weimer May 5, 2000

May 5, 2000

Just van Rossum <just(a)letterror.com> writes: > Good point. All this taken together still means to me that comparisons > between wide and narrow strings should take place at the character level, > which implies that coercion from narrow to wide is done at the character > level, without looking at the encoding. (Which in my book in turn still > implies that as long as we're talking about Unicode, narrow strings are > effectively Latin-1.) Sorry for jumping in, I've … [View More]

1 0

Moving Unicode debate to i18n-sig@python.org
by Guido van Rossum May 5, 2000

May 5, 2000

I've moved all my responses to the Unicode debate to the i18n-sig mailing list, where it belongs. Please don't cross-post any more. If you're interested in this issue but aren't subscribed to the i18n-sig list, please subscribe at http://www.python.org/mailman/listinfo/i18n-sig/. To view the archives, go to http://www.python.org/pipermail/i18n-sig/. See you there! --Guido van Rossum (home page: http://www.python.org/~guido/)

1 0

Re: [I18n-sig] Unicode strings: an alternative
by Just van Rossum May 5, 2000

May 5, 2000

(Boy, is it quiet here all of a sudden ;-) Sorry for the duplication of stuff, but I'd like to reiterate my points, to separate them from my implementation proposal, as that's just what it is: an implementation detail. These things are important to me: - get rid of the Unicode-ness of wide strings, in order to - make narrow and wide strings as similar as possible - implicit conversion between narrow and wide strings should happen purely on the basis of the character codes; no assumption … [View More]

6 8

Re: [XML-SIG] Re: [I18n-sig] Re: [Python-Dev] Unicode debate
by ht＠cogsci.ed.ac.uk May 5, 2000

May 5, 2000

Guido van Rossum <guido(a)python.org> writes: <snip/> > My ASCII proposal is a compromise that tries to be fair to both uses > for strings. Introducing byte arrays as a more fundamental type has > been on the wish list for a long time -- I see no way to introduce > this into Python 1.6 without totally botching the release schedule > (June 1st is very close already!). I'd like to be able to move on, > there are other important things still to be added to 1.6 (… [View More]

5 4

Re: [XML-SIG] Re: Unicode debate
by ht＠cogsci.ed.ac.uk May 5, 2000

May 5, 2000

"Fredrik Lundh" <fredrik(a)pythonware.com> writes: > Guido van Rossum <guido(a)python.org> wrote: > > Thanks -- but that's really Tcl's fault, since the only way to get > > character data *into* Tcl (or out of it) is through the UTF-8 > > encoding. > > from http://dev.scriptics.com/man/tcl8.3/TclLib/StringObj.htm > > Tcl_NewUnicodeObj(Tcl_UniChar* unicode, int numChars) > > Tcl_NewUnicodeObj and Tcl_SetUnicodeObj create a new > … [View More]

1 0

Re: [XML-SIG] Re: [I18n-sig] Re: [Python-Dev] Unicode debate
by ht＠cogsci.ed.ac.uk May 5, 2000

May 5, 2000

Guido van Rossum <guido(a)python.org> writes: > > I think I hear a moderate consensus developing that the 'ASCII > > proposal' is a reasonable compromise given the time constraints. But > > let's not fail to come back to this ASAP -- it _really_ narcs me that > > every time I load XML into my Python-based editor I'm going to convert > > large amounts of wide-string data into UTF-8 just so Tk can convert it > > back to wide-strings in order to display … [View More]

1 0

FW: weird bug in test_winreg
by Mark Hammond May 4, 2000

May 4, 2000

Oops - I didnt notice the CC - a copy of what I sent to Guido: -----Original Message----- From: Mark Hammond [mailto:mhammond@skippinet.com.au] Sent: Thursday, 4 May 2000 11:13 AM To: Guido van Rossum Subject: RE: weird bug in test_winreg Hah - I was just thinking about this this myself. If I wasnt waiting 24 hours, I would have beaten you to the test_fork1 patch :-) However, there is something bad going on. If you remove your test_fork1 patch, and run it from regrtest (_not_ stand alone) … [View More]

2 2

Minimal (single-file) Python?
by gvwilson＠nevex.com May 4, 2000

May 4, 2000

Hi. Has anyone ever built, or thought about building, a single-file Python, in which all the "basic" capabilities are included in a single executable (where "basic" means "can do as much as the Bourne shell")? Some of the entries in the Software Carpentry competition would like to be able to bootstrap from as small a starting point as possible. Thanks, Greg p.s. I don't think this is the same problem as moving built-in features of Python into optionally-loaded libraries, as some of the … [View More]

1 0

Unicode strings: an alternative
by Just van Rossum May 4, 2000

May 4, 2000

Today I had a relatively simple idea that unites wide strings and narrow strings in a way that is more backward comatible at the C level. It's quite possible this has already been considered and rejected for reasons that are not yet obvious to me, but I'll give it a shot anyway. The main concept is not to provide a new string type but to extend the existing string object like so: - wide strings are stored as if they were narrow strings, simply using two bytes for each Unicode character. - … [View More]

4 6