json decoder speedups, any time left for 2.6?
I'm out of town this week for a conference (ICFP/CUFP in Victoria) and my hotel's connection has been bad enough such that I can't get any Real Work done so I've managed to hammer on the json library's decoding quite a bit instead. I just released simplejson 1.9.3 which improves decoding performance by about 2x and I've got some more changes along the way in trunk for 1.9.4 that will increase it even further (over 3x my original 1.9.2 benchmark perf). How much time do I have left to get this into Python 2.6? FWIW the changes are all on the Python side, no C code has been harmed (yet). The test suite still passes of course. -bob
Bob Ippolito wrote:
How much time do I have left to get this into Python 2.6?
Zero I'm afraid - with rc1 out, it's release blocker bugs only. Anything which can be deferred to the 2.6.1 release without causing any major harm is definitely out - and while a 2x speedup is very nice, it isn't something to be squeezing in post-rc1. Still, that should make for a nice incremental improvement when 2.6.1 rolls around. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia --------------------------------------------------------------- http://www.boredomandlaziness.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Sep 24, 2008, at 5:47 AM, Nick Coghlan wrote:
Bob Ippolito wrote:
How much time do I have left to get this into Python 2.6?
Zero I'm afraid - with rc1 out, it's release blocker bugs only. Anything which can be deferred to the 2.6.1 release without causing any major harm is definitely out - and while a 2x speedup is very nice, it isn't something to be squeezing in post-rc1.
Still, that should make for a nice incremental improvement when 2.6.1 rolls around.
I concur. - -Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSNo9PHEjvBPtnXfVAQKJZQP/dGcmjO7P9RKIcYsFAwmDdVlCfn2WiGQt WlNgJiIq5tnq4WSyRQMNeDAWZfheq/vHqlDMU6OunIz5jcv0cVNmKkfcW+pTl9Yc 7Ur6mejeAUYrg2jHkimhMv0wpX/lJ+7CrWD1c+3PK+tQD0GgWfen+/xzCkGqlbc8 ompjNE1WPPs= =Jm8s -----END PGP SIGNATURE-----
On Wed, Sep 24, 2008 at 6:14 AM, Barry Warsaw <barry@python.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Sep 24, 2008, at 5:47 AM, Nick Coghlan wrote:
Bob Ippolito wrote:
How much time do I have left to get this into Python 2.6?
Zero I'm afraid - with rc1 out, it's release blocker bugs only. Anything which can be deferred to the 2.6.1 release without causing any major harm is definitely out - and while a 2x speedup is very nice, it isn't something to be squeezing in post-rc1.
Still, that should make for a nice incremental improvement when 2.6.1 rolls around.
I concur.
Ok, no problem. The speedup is about 3x now on the trunk ;) I think that further optimization will require some more C hacking, but 2.6.1 should give me plenty of time to get around to some of that. -bob
Meanwhile, can you please release (wherever you normally release things;-) the pure-Python version as well? I'd like to play around with it in Google App Engine opensource sandboxes (e.g., cfr. gae-json-rest -- I'll be delighted to add you to that project if you want of course;-) and that requires Python 2.5 and only pure-Python add-ons... thanks! Alex On Wed, Sep 24, 2008 at 8:08 AM, Bob Ippolito <bob@redivi.com> wrote:
On Wed, Sep 24, 2008 at 6:14 AM, Barry Warsaw <barry@python.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Sep 24, 2008, at 5:47 AM, Nick Coghlan wrote:
Bob Ippolito wrote:
How much time do I have left to get this into Python 2.6?
Zero I'm afraid - with rc1 out, it's release blocker bugs only. Anything which can be deferred to the 2.6.1 release without causing any major harm is definitely out - and while a 2x speedup is very nice, it isn't something to be squeezing in post-rc1.
Still, that should make for a nice incremental improvement when 2.6.1 rolls around.
I concur.
Ok, no problem. The speedup is about 3x now on the trunk ;) I think that further optimization will require some more C hacking, but 2.6.1 should give me plenty of time to get around to some of that.
-bob _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com
http://pypi.python.org/pypi/simplejson The _speedups module is optional. On Wed, Sep 24, 2008 at 8:42 AM, Alex Martelli <aleaxit@gmail.com> wrote:
Meanwhile, can you please release (wherever you normally release things;-) the pure-Python version as well? I'd like to play around with it in Google App Engine opensource sandboxes (e.g., cfr. gae-json-rest -- I'll be delighted to add you to that project if you want of course;-) and that requires Python 2.5 and only pure-Python add-ons... thanks!
Alex
On Wed, Sep 24, 2008 at 8:08 AM, Bob Ippolito <bob@redivi.com> wrote:
On Wed, Sep 24, 2008 at 6:14 AM, Barry Warsaw <barry@python.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Sep 24, 2008, at 5:47 AM, Nick Coghlan wrote:
Bob Ippolito wrote:
How much time do I have left to get this into Python 2.6?
Zero I'm afraid - with rc1 out, it's release blocker bugs only. Anything which can be deferred to the 2.6.1 release without causing any major harm is definitely out - and while a 2x speedup is very nice, it isn't something to be squeezing in post-rc1.
Still, that should make for a nice incremental improvement when 2.6.1 rolls around.
I concur.
Ok, no problem. The speedup is about 3x now on the trunk ;) I think that further optimization will require some more C hacking, but 2.6.1 should give me plenty of time to get around to some of that.
-bob _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com
simplejson 2.0.0 is now released which is about as optimized as I can be bothered to make it. It's about 4x faster than cPickle for encoding and just a little slower and decoding, which is good enough for now ;) The pure Python source is much uglier now (to avoid global lookups, etc.), but also several times faster than it was. http://pypi.python.org/pypi/simplejson One of the optimizations I made probably isn't good for Py3k, it will return ASCII strings as str objects instead of converting to unicode, but that shouldn't be too much work to port (though I haven't looked at the current _json.c port for Py3k). I also converted over to using Sphinx documentation, which was nice because I was able to just re-use the docs that were already in Python trunk after changing the module name around. All of the work should be easy to merge back into trunk so I'll try and take care of that quickly after Python 2.6 is released. On Wed, Sep 24, 2008 at 9:02 AM, Bob Ippolito <bob@redivi.com> wrote:
http://pypi.python.org/pypi/simplejson
The _speedups module is optional.
On Wed, Sep 24, 2008 at 8:42 AM, Alex Martelli <aleaxit@gmail.com> wrote:
Meanwhile, can you please release (wherever you normally release things;-) the pure-Python version as well? I'd like to play around with it in Google App Engine opensource sandboxes (e.g., cfr. gae-json-rest -- I'll be delighted to add you to that project if you want of course;-) and that requires Python 2.5 and only pure-Python add-ons... thanks!
Alex
On Wed, Sep 24, 2008 at 8:08 AM, Bob Ippolito <bob@redivi.com> wrote:
On Wed, Sep 24, 2008 at 6:14 AM, Barry Warsaw <barry@python.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Sep 24, 2008, at 5:47 AM, Nick Coghlan wrote:
Bob Ippolito wrote:
How much time do I have left to get this into Python 2.6?
Zero I'm afraid - with rc1 out, it's release blocker bugs only. Anything which can be deferred to the 2.6.1 release without causing any major harm is definitely out - and while a 2x speedup is very nice, it isn't something to be squeezing in post-rc1.
Still, that should make for a nice incremental improvement when 2.6.1 rolls around.
I concur.
Ok, no problem. The speedup is about 3x now on the trunk ;) I think that further optimization will require some more C hacking, but 2.6.1 should give me plenty of time to get around to some of that.
-bob _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/aleaxit%40gmail.com
participants (4)
-
Alex Martelli -
Barry Warsaw -
Bob Ippolito -
Nick Coghlan