[Python-Dev] Status of json (simplejson) in cpython

[Vinay Sajip]

Sandro Tosi <sandro.tosi <at> gmail.com> writes:

> The version we have in cpython of json is simplejson 2.0.9 highly
> patched (either because it was converted to py3k, and because of the
> normal flow of issues/bugfixes) while upstream have already released
> 2.1.13 .

I think you mean 2.1.3?
 
> Their 2 roads had diverged a lot, and since this blocks any further
> update of cpython's json from upstream, I'd like to close this gap.
> 
> This isn't exactly an easy task, and this email is more about a
> brainstorming on the ways we have to achieve the goal: being able to
> upgrade json to 2.1.13.
> 
> Luckily, upstream is receptive for patches, so part of the job is to
> forward patches written for cpython not already in the upstream code.
> 
> But how am I going to do this? let's do a brain-dump:
> 
> - the history goes back at changeset f686aced02a3 (May 2009, wow) when
> 2.0.9 was merged on trunk
> - I can navigate from that CS up to tip, and examine the diffs and see
> if they apply to 2.1.3 and prepare a set of patches to be forwarded
> - part of those diffs is about py3k conversion, that probably needs to
> be extended to other part of the upstream code not currently in
> cpython. For those "new" code parts, do you have some guides about
> porting a project to py3k? it would be my first time and other than
> building it and running it with python3 i don't know what to do :)
> - once (and if :) I reach the point where I've all the relevant
> patches applied on 2.1.3 what's the next step?

If it is generally considered desirable to maintain some synchrony between
simplejson and stdlib json, then since Bob has stated that he no interest in
Python 3, it may be better to:

1. Convert the simplejson codebase so that it runs on both Python 2 and 3
(without running 2to3 on it). Once this is done, if upstream accepts these
changes, ongoing maintenance will be fairly simple for upstream, and changes
only really need to consider exception and string/byte literal syntax, for the
most part.
2. Merge this new simplejson with stdlib json for 3.3.

I looked at step 1 a few weeks ago and have made some progress with it. I've
just forked simplejson on Github and posted my changes to my fork:

https://github.com/vsajip/simplejson

All 136 tests pass on Python 2.7 (just as a control/sanity check), and on Python
3.2, there are 4 failures and 12 errors - see complete results at

https://gist.github.com/923019

I haven't looked at the C extension yet, just the Python code. I believe most of
the test failures will be down to string literals in the tests which should be
bytes, e.g. test_unicode.py:test_ensure_ascii_false_bytestring_encoding.

So, it looks quite encouraging, and if you think my approach has merit, please
take a look at my fork, and give feedback/join in!

Note that I used the same approach when porting pip/virtualenv to Python 3,
which seems to have gone quite smoothly :-)

Regards,


Vinay Sajip