[Tutor] Fwd: Re: Using files to read data

[Peter Otten]

Alan Gauld via Tutor wrote:

> Forwarding to list
> 
> Please always use ReplyAll or ReplyList when responding to list mail.
> 
> 
> 
> -------- Forwarded Message --------
> 
> i apologize.  i guess it didn’t attach correctly.
> my issue is how do i get it out of my file and use it.
> 
> *the json file, its only about a fifth of it but it should serve its
> purpose*

> [
> 0.9888888888888889,
> 0.011111111111111112,
> "Mon Jun 26 20:37:34 2017"
> ]
> [
> 0.9777777777777777,
> 0.022222222222222223,
> "Mon Jun 26 20:37:34 2017"
> ]

The problem with this data is that it's not proper json which allows only 
one toplevel data structure, e. g.

[
> [
> 0.9888888888888889,
> 0.011111111111111112,
> "Mon Jun 26 20:37:34 2017"
> ]
,
> [
> 0.9777777777777777,
> 0.022222222222222223,
> "Mon Jun 26 20:37:34 2017"
> ]
]

If you stick to that file format you need to split it into parts that are 
valid json before you can process them with the standard library's json 
module.

If you can rely on your records always consisting of five lines the 
following should work


import json

records = []

def read_five_lines(f):
    return "".join(f.readline() for i in range(5))

with open("xyz_save.json") as f:
    while True:
        chunk = read_five_lines(f)
        if not chunk:  # empty string --> reached end of file
            break
        records.append(json.loads(chunk))

print(records)

Another option would be to change your writing script so that it writes one 
record per line, e. g.

[0.24444444444444446, 0.8555555555555556, "Tue Jun 27 14:21:44 2017"]
[-0.29333333333333333, 1.906666666666667, "Tue Jun 27 14:21:44 2017"]

This can be achieved by omitting the indent=0 argument in your json.dump() 
calls. Then your reader can be simplified:

import json

records = []

with open("xyz_save.json") as f:
    for line in f:
        records.append(json.loads(line))

print(records)



>     if heading in range(1, 91):

Did you know that

>>> 1.5 in range(1, 91)
False

? If you want to accept non-integral values you better write the above as

if 0 <= heading < 90:  # swap operators if want to exclude the lower 
                       # and include the upper bound
    ...

or similar.

>         north = heading
>         east = 90 - heading
>         y = (north / 90) * forward_time
>         x = (east / 90) * forward_time
>         now_time = time.ctime()
>         
>         xyz = [x, y, now_time]
>         
>         xyz_save = "xyz_save.json"
> 
> 
>         with open(xyz_save, "a") as stamp:
>             json.dump(xyz, stamp, indent=0)
>             stamp.write("\n")
> 
>         return xyz
>         
>     elif heading in range(91, 181):
>         east = heading - 90
>         south = 180 - heading
>         y = (south / 90) * forward_time
>         x = (east / -90) * forward_time
>         now_time = time.ctime()
>         
>         xyz = [x, y, now_time]
>         
>         xyz_save = "xyz_save.json"
> 
> 
>         with open(xyz_save, "a") as stamp:
>             json.dump(xyz, stamp, indent=0)
>             stamp.write("\n")
> 
>         return xyz

There's *a* *lot* of repetition here. At the very least you should move the 
parts that are completely identical to the end of the if ... elif ... chain:

    if heading in range(1, 91):
        north = heading
        east = 90 - heading
        y = (north / 90) * forward_time
        x = (east / 90) * forward_time
    elif heading in range(91, 181):
        east = heading - 90
        south = 180 - heading
        y = (south / 90) * forward_time
        x = (east / -90) * forward_time
 ...

    else: 
        # handling the case where no condition matches
        print("unhandled heading", heading, file=sys.stderr)
        return

    now_time = time.ctime()
    xyz = [x, y, now_time]
    xyz_save = "xyz_save.json"

    with open(xyz_save, "a") as stamp:
        json.dump(xyz, stamp)
        stamp.write("\n")

    return xyz

PS:

>         north = heading
>         east = 90 - heading
>         y = (north / 90) * forward_time
>         x = (east / 90) * forward_time

I wonder whether there should be a sin() and cos() somewhere...