Suggestions on storing, caching, querying json
Sayth Renshaw
flebber.crue at gmail.com
Fri Oct 6 05:21:50 EDT 2017
On Thursday, 5 October 2017 15:13:43 UTC+11, Sayth Renshaw wrote:
> HI
>
> Looking for suggestions around json libraries. with Python. I am looking for suggestions around a long term solution to store and query json documents across many files.
>
> I will be accessing an api and downloading approx 20 json files from an api a week. Having downloaded this year I have over 200 files already. So it will grow at a reasonable rate.
>
> What I have initially done is store them into a mongo db. Now I am wondering if this is useful or prudent since other than querying the json I wont have much use of other mongo features.
>
> When querying the json files though queries will utilise multiple json files at once, not just retrieving a single record. The usage is for data analysis.
>
> Is there a good json storage option, with caching and optimal querying etc.
>
> Regarding querying I did find a library for json searching called ObjectPath written in Python http://objectpath.org/reference.html
>
> Looking to leverage your experience.
>
> Cheers
>
> Sayth
There is a new extension for redis ReJson and redis-py for using redis and python as a json store. http://rejson.io/ and https://github.com/andymccurdy/redis-py. Not sure if this has much more upside than mongo other than having a more fmailiar query language like JsonPath http://rejson.io/path/
Cheers
Sayth
More information about the Python-list
mailing list