[BangPypers] JSON PARSER

Prashant Gaur 91prashantgaur at gmail.com
Fri Mar 21 11:05:23 CET 2014


EDIT  :

import simplejson as json
from collections import defaultdict

def duplicate_key_lookup(ordered_pairs):
     """Convert duplicate keys values to lists."""
     # read all values into lists
     d = defaultdict(list)
     for k, v in ordered_pairs:
         d[k].append(v)
     # unpack lists that have only 1 item
     for k, v in d.items():
         if len(v) == 1:
             d[k] = v[0]
      return dict(d)

>>> json.loads('''{"test":"hi","test":"bye", "test1": "tata"}''',
object_pairs_hook=duplicate_key_lookup)
{'test': ['hi', 'bye'], 'test1': 'tata'}



On Fri, Mar 21, 2014 at 3:34 PM, Prashant Gaur <91prashantgaur at gmail.com>wrote:

> Hello Lokesh ,
>
> as we know json.loads return data in form of dict and dict can never have
> same keys .
>
> so we can do one thing and that is to return a list of all values which
> are having same names.
>
> import simplejson as json
> from collections import defaultdict
>
> def duplicate_key_lookup(ordered_pairs):
>     """Convert duplicate keys values to lists."""
>     # read all values into lists
>      d = defaultdict(list)
>      for k, v in ordered_pairs:
>          d[k].append(v)
>          # unpack lists that have only 1 item
>      for k, v in d.items():
>          if len(v) == 1:
>              d[k] = v[0]
>       return dict(d)
>
> >>> json.loads('''{"test":"hi","test":"bye", "test1": "tata"}''',
> object_pairs_hook=multidict)
> {'test': ['hi', 'bye'], 'test1': 'tata'}
>
>
> On Fri, Mar 21, 2014 at 3:22 PM, lokesh bobby <lokeshbobbys at yahoo.com>wrote:
>
>> Hi Jayanth/Prashant,
>>
>> Either "the duplicates to be loaded" or "An ouput of all the duplicate
>> key names in a JSON file" should be fine for me :-)
>>
>> NOTE: JSON file consits of more than 2 lakhs LOC
>>
>> Lokesh
>>
>>
>>   On Friday, 21 March 2014 3:17 PM, Jayanth Koushik <jnkoushik at gmail.com>
>> wrote:
>>   Hi Prashant
>>
>> I think he wants the duplicates to be loaded, not cause an exception.
>>
>> Jayanth
>>
>>
>> On Fri, Mar 21, 2014 at 3:15 PM, Prashant Gaur <91prashantgaur at gmail.com>wrote:
>>
>> Hi Lokesh,
>>
>> we can pass lookup while parsing your json file which will make sure that
>> name is repetitive or not .
>>
>> import json
>>
>> def duplicate_checking_hook(pairs):
>>      ''' lookup for duplicate names'''
>>     result = dict()
>>     for key, val in pairs:
>>         if key in result:
>>             raise KeyError('Duplicate Key specified: %s % key)
>>         result[key] = val
>>     return result
>>
>> >>> json.loads('''{"test":"hi","test":"bye"}''',
>> object_pairs_hook=duplicate_checking_hook)
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>>   File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
>>     return cls(encoding=encoding, **kw).decode(s)
>>   File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
>>     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
>>   File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
>>     obj, end = self.scan_once(s, idx)
>>   File "<stdin>", line 5, in duplicate_checking_hook
>> KeyError: u'Duplicate key specified: a'
>>
>> While
>>
>> >>> json.loads('''{"test":"hi","test1":"bye"}''',
>> object_pairs_hook=duplicate_checking_hook)
>> {u'test': u'hi', u'test1': u'bye'}
>> >>>
>>
>>
>>
>>  def dupe_checking_hook(pairs):
>>     result = dict()
>>     for key,val in pairs:
>>         if key in result:
>>             raise KeyError("Duplicate key specified: %s" % key)
>>         result[key] = val
>>     return result
>>
>>
>>
>>
>>
>> On Fri, Mar 21, 2014 at 2:59 PM, lokesh bobby <lokeshbobbys at yahoo.com>wrote:
>>
>> Hi Jayanth,
>>
>> Ideally speaking a JSON shouldn't be created with repetitive key names.
>> But manually it is possible that a proper JSON file can be appended with a
>> duplicate key. We need to catch that duplicate key. If we are going to use
>> json.load(), the repetitive keys of the JSON file wont get loaded. Only the
>> last occurrence of the key will be there in that loaded data.
>>
>> Instead I want that entire data in JSON file should be loaded. Hoping
>> that you got the point what I want :-)
>>
>> Thanks,
>> Lokesh.
>>
>>
>>
>>
>>
>> On Friday, 21 March 2014 2:45 PM, Jayanth Koushik <jnkoushik at gmail.com>
>> wrote:
>>
>> Hi Lokesh
>>
>>
>> The 'problem' that you talk about isn't really a problem. Since the JSON
>> specification does not say what is to be done for repeated names, it is up
>> to the implementation to decide. What is your requirement for handling
>> repeated names?
>>
>>
>> Jayanth
>>
>>
>>
>>
>> On Fri, Mar 21, 2014 at 2:30 PM, lokesh bobby <lokeshbobbys at yahoo.com>
>> wrote:
>>
>> Hi Noufal,
>> >
>> >Thanks for your reply. I am not looking for loading the JSON file. There
>> is a limitation in it. Go thru the links
>> >
>> >
>> http://docs.python.org/2/library/json.html#repeated-names-within-an-object
>> >
>> http://docs.python.org/3.2/library/json.html#repeated-names-within-an-object
>> >
>> >In order to get rid of that problem, I am looking for some JSON stream
>> parsers.
>> >
>> >Thanks,
>> >Lokesh.
>> >
>> >
>> >
>> >
>> >On Friday, 21 March 2014 2:09 PM, Noufal Ibrahim KV <
>> noufal at nibrahim.net.in> wrote:
>> >
>> >On Fri, Mar 21 2014, lokesh bobby wrote:
>> >
>> >
>> >> Hi ALL,
>> >>
>> >> Can you share your thoughts on how to parse a JSON file by using
>> >> python?
>> >
>> >import json
>> >
>> >with open("data.json") as f:
>> >     json.load(f)
>> >
>> >
>> >[...]
>> >
>> >
>> >--
>> >Cordially,
>> >Noufal
>> >http://nibrahim.net.in
>> >_______________________________________________
>> >BangPypers mailing list
>> >BangPypers at python.org
>> >https://mail.python.org/mailman/listinfo/bangpypers
>> >
>> _______________________________________________
>> BangPypers mailing list
>> BangPypers at python.org
>> https://mail.python.org/mailman/listinfo/bangpypers
>>
>>
>>
>>
>> --
>> Prashant Gaur
>>
>> Mobile : +91 9717353657
>> http://gaurprashant.blogspot.in/
>> http://stackoverflow.com/users/1850358/prashant-gaur
>> http://www.about.me/prashantgaur/
>>
>>
>>
>>
>>
>
>
> --
> Prashant Gaur
>
> Mobile : +91 9717353657
> http://gaurprashant.blogspot.in/
> http://stackoverflow.com/users/1850358/prashant-gaur
> http://www.about.me/prashantgaur/
>
>


-- 
Prashant Gaur

Mobile : +91 9717353657
http://gaurprashant.blogspot.in/
http://stackoverflow.com/users/1850358/prashant-gaur
http://www.about.me/prashantgaur/


More information about the BangPypers mailing list