[Python-3000] C API for ints and strings

[Nicholas Bastin]

On 9/9/07, Jim Jewett <jimjjewett at gmail.com> wrote:
> On 9/8/07, Nicholas Bastin <nick.bastin at gmail.com> wrote:
> > On 9/8/07, "Martin v. Löwis" <martin at v.loewis.de> wrote:
> > > > Speaking of PyLong, and its' minor awkwardness to work with in C (you
> > > > either have to convert to another multiple-precision type through a
> > > > string, or use Python's arithmetic operators directly), was there any
> > > > thought given to using something like GPM's mpz_t as the backing data
> > > > type?
>
> > Would anyone be opposed to rehosting PyLong on top of GMP?
>
> (1)  If there are concerns about the RCA attribution license, I would
> expect much greater concerns about LGPL.

Maybe, but I'd rather have a technical discussion than a licensing
discussion.  If GMP doesn't stand up for technical reasons, then the
licensing discussion was a waste of time without resolving whether it
would be a good technical decision or not.

> (2)  License aside, does it really solve the problem you had about
> needing to convert or use Python's arithmetic operations?   At first
> glance, it looks like you would still have the same problem, except
> that you would need to use the GMP functions instead of the python
> functions.

Yes, but the GMP function set is much richer than the Python one, and
more efficient.  GMP is in fact the thing I most convert PyLong to
(via a string, which is, as you might imagine, not that efficient).
Obviously if we're going to support numbers larger than the host
language (in this case, C) natively supports, there's going to be some
other API involved.

> (3)  Is it stable enough?  I know it has been developed since 1991,
> but they seem to focus on high performance for truly huge numbers.  I
> suspect the vast majority of python programs would perform fine if
> they were limited to C ints, and so the extra costs may not be worth
> it.

In a little test, integer math (not-long-requiring) in 3.0 is 2.3x
slower than the same integer math in 2.6.  Here is my test code:

inttest.py:
def int_test(rounds):
  index = 0
  while index < rounds:
    foo = 0
    while foo < 10000000:
      foo += 1
      .... (above line repeated 100 times)

    index += 1

3.0:  python Lib\timeit.py "import inttest; inttest.int_test (5)"
10 loops, best of 3: 6.01 sec per loop

2.6:  python Lib\timeit.py "import inttest; inttest.int_test (5)"
10 loops, best of 3: 2.64 sec per loop

I welcome other benchmarks if people think there's something
fundamentally wrong with my test.

> It doesn't work on the Intel Macintoshes, and the workarounds are so
> ugly that they won't be applied to the trunk.

This is clearly a deal killer, thanks for pointing that out.

I would however continue to ask the general question - do we really
want to maintain our own arbitrary precision math library (which we
now use exclusively)?  Who is committing to optimizing the performance
of PyLong?

--
Nick