longinteger on Cray J90

[Tim Peters]

[posted & mailed]

[Marc Poinot]
> I'm trying to use Python on a J90.

Poor Marc!

> This Cray, and its folks, has 8 bytes
> size shorts, int, long, long long, etc...
> Looks like some hardware discount ;) ?
>
> I have a 8 bytes concern with longobject.c
> which manipulates heavily bit masks.
>
> I found that the carry (add) and borrow (sub)
> are not "masked" correctly as shorts.

I don't know what this means, alas, but since you're likely the only person
on c.l.py with access to a J90, you're going to have to work hard to be
clear.

> Thus, it looks like some of the asserts are
> not relevants (i.e. assert(borrow == 0) ),
> but not only the asserts.

The asserts are certainly relevant.  If the asserts are triggering,
something is wrong with the code.

> As soon as your long object is larger than SHIFT,
> at least the borrow/carry is not "cleared" correctly.
>
> Python 1.5.2 (#16, May 22 2000, 12:14:02) [C] on unicos10
> Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
> >>> a=01000000000000000

Is there some reason this is in octal?  And/or some reason 2**45 is
especially interesting?

> >>> a-1
> 35184372088831
> >>> b=long(a)
> >>> b
> 35184372088832L
> >>> b-1
> Assertion failed: borrow == 0, file longobject.c, line 1120
> Abort(coredump)

Well, *that's* not good <wink>.

> I've read some emails from Konrad H. and Tim P.
> about that. K.Hinsen has problems with regexp.
> T.Peters suggested the use of a macro
> to mask the shorts.

That was different:  the old regex code (which Guido didn't write) assumed
that sizeof(short) == 2 exactly, and that's a bad assumption.  Python's
longint code *intended* to assume only that

1. sizeof(unsigned short) >= 2

2. sizeof(unsigned long) >= 4

There is *no* intended assumption that sizeof(unsigned long) >=
2*sizeof(unsigned short), or any of the other bad assumptions you're
thinking of.  So, given that this code doesn't work for you, there are two
likely possibilities:

A. The compiler is generating bad code.  Did you try compiling Python
   *without* optimization?  If not, try that.

B. It's a shallow bug in a line or two of longobject.c, easy to track down
   if you run under a debugger on a machine where it fails, but could be
   impossible to track down via email without access to a failing platform.

> But such a use seems difficult as the long object representation
> has a digit typedef, I have to find back the use of this type in the
> whole Python code... (I am wrong, K.P. ?).

The "digit" typedef is used only in longintrepr.h and longobject.c.  If you
stop being paralyzed long enough to look at them <wink>, you'll find that
almost all uses are already *obviously* properly masked; e.g.,

			v->ob_digit[i] = (digit) (t & MASK);

> The description in Include/longinterp.h gives some
> details about what the short/digit/wdigit types should
> be able to do. But in my case, I wonder how to make
> a wdigit able to handle twice the size of a digit?

That's not the assumption the code makes.  A twodigit thing needs to be able
to hold an int no bigger than MASK << SHIFT, and MASK is just BASE-1, and
BASE is just 1<<SHIFT, and SHIFT is just 15:  your twodigit is already wide
enough.  You didn't happen to change the SHIFT #define in the code to
something larger than 15?  If you did, that's why it's blowing up.

Ponder ... I think I see a problem here (in x_sub):

	for (i = 0; i < size_b; ++i) {
		/* The following assumes unsigned arithmetic
		   works module 2**N for some N>SHIFT. */
		borrow = a->ob_digit[i] - b->ob_digit[i] - borrow;
		z->ob_digit[i] = borrow & MASK;
		borrow >>= SHIFT;
		borrow &= 1; /* Keep only one sign bit */
	}
	for (; i < size_a; ++i) {
		borrow = a->ob_digit[i] - borrow;
		z->ob_digit[i] = borrow & MASK;
		borrow >>= SHIFT;
	}
	assert(borrow == 0);

Note that the second "for" loop does not have the

		borrow &= 1; /* Keep only one sign bit */

line that's at the bottom of the first "for" loop.  Try copying that line
into the bottom of the second "for" loop too:

	for (; i < size_a; ++i) {
		borrow = a->ob_digit[i] - borrow;
		z->ob_digit[i] = borrow & MASK;
		borrow >>= SHIFT;
		borrow &= 1; /* Keep only one sign bit */  ADD THIS LINE ******
	}

Without that, if the first line ever computes a negative value, "borrow" at
the bottom on your machine will retain 64-15 = 49 copies of the sign bit.
On machines where sizeof(short) == 2, it retains just 16-15=1 sign bit.
Fixing that may well be all there is to this -- but I can't test it for you.

the-line-that's-broken-is-always-the-line-closest-to-the-assert<wink>-ly
    y'rs  - tim