Mailman 3 Re: [Python-Dev] [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed - Python-Dev

Re: [Python-Dev] [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed

Victor Stinner

21 Mar 2012 21 Mar '12

10:22 p.m.

...
http://hg.python.org/cpython/rev/7355550d5357 changeset: 75850:7355550d5357 user: Stefan Krah date: Wed Mar 21 18:25:23 2012 +0100 summary: Issue #7652: Integrate the decimal floating point libmpdec library to speed up the decimal module. Performance gains of the new C implementation are between 12x and 80x, depending on the application.

Congrats Stefan! And thanks for the huge chunk of code. Victor

Show replies by date

Stefan Krah

22 Mar 22 Mar

12:45 p.m.

New subject: [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed

Victor Stinner wrote:

...

...
...
Issue #7652: Integrate the decimal floating point libmpdec library to speed up the decimal module. Performance gains of the new C implementation are between 12x and 80x, depending on the application.

Congrats Stefan! And thanks for the huge chunk of code.

Thanks, much appreciated. I'll take the opportunity to thank you in return for the gigantic amount of work you've done on Python in the past year! Stefan Krah

Dirkjan Ochtman

12:58 p.m.

New subject: [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed

On Wed, Mar 21, 2012 at 23:22, Victor Stinner wrote:

...

...
...
http://hg.python.org/cpython/rev/7355550d5357 changeset: 75850:7355550d5357 user: Stefan Krah date: Wed Mar 21 18:25:23 2012 +0100 summary: Issue #7652: Integrate the decimal floating point libmpdec library to speed up the decimal module. Performance gains of the new C implementation are between 12x and 80x, depending on the application.

As a Python user, this looks really cool, thanks! As a packager, is the libmpdec library used elsewhere? For Gentoo, we generally prefer to package libraries separately and have Python depend on them. From the site, it seems like you more or less wrote libmpdec for usage in Python, but if it's general-purpose and actually used in other software, it would be nice if Python grew a configure option to make it use the system libmpdec. Cheers, Dirkjan

Stefan Krah

23 Mar 23 Mar

8:26 a.m.

New subject: [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed

Dirkjan Ochtman wrote:

...

As a packager, is the libmpdec library used elsewhere? For Gentoo, we generally prefer to package libraries separately and have Python depend on them. From the site, it seems like you more or less wrote libmpdec for usage in Python, but if it's general-purpose and actually used in other software, it would be nice if Python grew a configure option to make it use the system libmpdec.

libmpdec was actually written before the module. It's general purpose and it fully implements the specification. I'm only aware of in-house usage. Someone has tried to submit a libmpdec package to OpenSUSE, but it was rejected with the claim that there already exists a package with the name. I think the claim is false: There is a libmpcdec package, where "cdec" presumably stands for "codec". I'll add the --with-system-libmpdec option with the caveat that changes will probably make it first into the libmpdec shipped with Python, see also: http://bugs.python.org/issue7652#msg155744 On the bright side, I don't expect many changes, since the specification is stable. Stefan Krah

Dirkjan Ochtman

8:41 a.m.

New subject: [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed

On Fri, Mar 23, 2012 at 09:26, Stefan Krah wrote:

...

I'll add the --with-system-libmpdec option with the caveat that changes will probably make it first into the libmpdec shipped with Python, see also:

http://bugs.python.org/issue7652#msg155744

Sounds good, thanks! Cheers, Dirkjan

Georg Brandl

22 Mar 22 Mar

7:46 p.m.

New subject: [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed

On 21.03.2012 23:22, Victor Stinner wrote:

...

...
...
http://hg.python.org/cpython/rev/7355550d5357 changeset: 75850:7355550d5357 user: Stefan Krah date: Wed Mar 21 18:25:23 2012 +0100 summary: Issue #7652: Integrate the decimal floating point libmpdec library to speed up the decimal module. Performance gains of the new C implementation are between 12x and 80x, depending on the application.

Congrats Stefan! And thanks for the huge chunk of code.

Seconded. This is the kind of stuff that will make 3.3 the most awesomest 3.x release ever (and hopefully convince people that it does make sense to port)... cheers, Georg

Amaury Forgeot d'Arc

9:44 p.m.

New subject: [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed

2012/3/22 Georg Brandl :

...

...
Congrats Stefan! And thanks for the huge chunk of code.

Seconded. This is the kind of stuff that will make 3.3 the most awesomest 3.x release ever (and hopefully convince people that it does make sense to port)...

On the other hand, porting PyPy to 3.3 will be more work ;-) Fortunately the libmpdec directory should be reusable as is. Nice work! -- Amaury Forgeot d'Arc

Stefan Krah

23 Mar 23 Mar

9:30 a.m.

New subject: [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed

Amaury Forgeot d'Arc wrote:

...

...
Seconded. ?This is the kind of stuff that will make 3.3 the most awesomest 3.x release ever (and hopefully convince people that it does make sense to port)...

On the other hand, porting PyPy to 3.3 will be more work ;-)

We've got to keep you on your toes, don't we? :)

...

Fortunately the libmpdec directory should be reusable as is. Nice work!

Thanks, also for helping out with the MutableMapping context. Stefan Krah

Stefan Krah

9:22 a.m.

New subject: [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed

Georg Brandl wrote:

...

...
...
...
Issue #7652: Integrate the decimal floating point libmpdec library to speed up the decimal module. Performance gains of the new C implementation are between 12x and 80x, depending on the application.

Congrats Stefan! And thanks for the huge chunk of code.

Seconded. This is the kind of stuff that will make 3.3 the most awesomest 3.x release ever (and hopefully convince people that it does make sense to port)...

Thanks! For cdecimal specifically I have the impression that 3.x is already used in the financial community, where web framework dependencies aren't an issue. On the web side, there seems to be a huge interest in speeding up database accesses, so let me evangelize again: Database applications using decimal will run 12x faster in 3.3. Stefan Krah

Victor Stinner

9:28 a.m.

New subject: [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed

By the way, how much faster is cdecimal? 72x or 80x? http://docs.python.org/dev/whatsnew/3.3.html#decimal Victor 2012/3/23 Stefan Krah :

...

Georg Brandl wrote:

...
...
...
...
Issue #7652: Integrate the decimal floating point libmpdec library to speed up the decimal module. Performance gains of the new C implementation are between 12x and 80x, depending on the application.

Congrats Stefan! And thanks for the huge chunk of code.

Seconded. This is the kind of stuff that will make 3.3 the most awesomest 3.x release ever (and hopefully convince people that it does make sense to port)...

Thanks! For cdecimal specifically I have the impression that 3.x is already used in the financial community, where web framework dependencies aren't an issue.

On the web side, there seems to be a huge interest in speeding up database accesses, so let me evangelize again: Database applications using decimal will run 12x faster in 3.3.

Stefan Krah

_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com

Stefan Krah

10:40 a.m.

New subject: [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed

Victor Stinner wrote:

...

By the way, how much faster is cdecimal? 72x or 80x? http://docs.python.org/dev/whatsnew/3.3.html#decimal

It really depends on the precision. Also, the performance of decimal.py depends on many other things in the Python tree, so it easily changes +-10%. Currently, decimal.py seems to be 10% faster than in 3.2, maybe because of the new string representation. The 80x is a ballpark figure for the maximum expected speedup for standard numerical floating point applications. factorial(1000) is 219x faster in _decimal, and with increasing precision the difference gets larger and larger. For huge numbers _decimal is also faster than int: factorial(1000000): _decimal, calculation time: 6.844487905502319 _decimal, tostr(): 0.033592939376831055 int, calculation time: 17.96010398864746 int, tostr(): ... still running ... Stefan Krah

Victor Stinner

25 Mar 25 Mar

11:03 p.m.

New subject: [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed

...

The 80x is a ballpark figure for the maximum expected speedup for standard numerical floating point applications.

Ok, but it's just surprising when you read the What's New document. 72x and 80x look to be inconsistent.

...

For huge numbers _decimal is also faster than int:

factorial(1000000):

_decimal, calculation time: 6.844487905502319 _decimal, tostr(): 0.033592939376831055

int, calculation time: 17.96010398864746 int, tostr(): ... still running ...

Hum, with a resolution able to store the result with all digits? If yes, would it be possible to reuse the multiply algorithm of _decimal (and maybe of other functions) for int? Or does it depend heavily on _decimal internal structures? Victor

Stefan Krah

26 Mar 26 Mar

10:47 p.m.

New subject: [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed

Victor Stinner wrote:

...

...
The 80x is a ballpark figure for the maximum expected speedup for standard numerical floating point applications.

Ok, but it's just surprising when you read the What's New document. 72x and 80x look to be inconsistent.

Yes, indeed, I'll reword that.

...

...
For huge numbers _decimal is also faster than int:

factorial(1000000):

_decimal, calculation time: 6.844487905502319 _decimal, tostr(): ?? ?? ?? ?? ??0.033592939376831055

int, calculation time: 17.96010398864746 int, tostr(): ... still running ...

Hum, with a resolution able to store the result with all digits?

Yes, you have to set context.prec (and emax) to the maximum values, then the result is an exact integer. The conversion to string is so fast because there is no complicated base conversion.

...

If yes, would it be possible to reuse the multiply algorithm of _decimal (and maybe of other functions) for int? Or does it depend heavily on _decimal internal structures?

Large parts of the Number Theoretic Transform could be reused, but there would be still quite a bit of work. Stefan Krah

Antoine Pitrou

23 Mar 23 Mar

11:10 a.m.

New subject: [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed

On Fri, 23 Mar 2012 10:22:55 +0100 Stefan Krah wrote:

...

Georg Brandl wrote:

...
...
...
...
Issue #7652: Integrate the decimal floating point libmpdec library to speed up the decimal module. Performance gains of the new C implementation are between 12x and 80x, depending on the application.

Congrats Stefan! And thanks for the huge chunk of code.

Seconded. This is the kind of stuff that will make 3.3 the most awesomest 3.x release ever (and hopefully convince people that it does make sense to port)...

Thanks! For cdecimal specifically I have the impression that 3.x is already used in the financial community, where web framework dependencies aren't an issue.

On the web side, there seems to be a huge interest in speeding up database accesses, so let me evangelize again: Database applications using decimal will run 12x faster in 3.3.

Are you sure it isn't 12.5x ? Regards Antoine.

Stefan Krah

11:47 a.m.

New subject: [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed

Antoine Pitrou wrote:

...

...
On the web side, there seems to be a huge interest in speeding up database accesses, so let me evangelize again: Database applications using decimal will run 12x faster in 3.3.

Are you sure it isn't 12.5x ?

Well, that was marketing for 3.3. Stefan Krah

4407

Age (days ago)

4412

Last active (days ago)

List overview

Download

14 comments

6 participants

participants (6)

Amaury Forgeot d'Arc
Antoine Pitrou
Dirkjan Ochtman
Georg Brandl
Stefan Krah
Victor Stinner

Re: [Python-Dev] [Python-checkins] cpython: Issue #7652: Integrate the decimal floating point libmpdec library to speed

tags

participants (6)