[Tutor] How to parse large files

jarod_v6 at libero.it jarod_v6 at libero.it
Sun Nov 1 13:31:03 EST 2015


My file have 1960607  rows but I don't understand why I'm not able to create a 
dictionary in fast way I try to use also gc.disable  but Not work.
I need to have dictionary but I have this erro:

with shelve.open("diz5") as db:
    with open("tmp1.txt") as instream:
        for line in instream:
            assert line.count("\t") == 1
            key, _tab, value = line.rstrip("\n").partition("\t")
            values = db.get(key) or set()
            values.add(value)
            db[key] = values

AttributeError                            Traceback (most recent call last)
<ipython-input-3-f1c2a78eeb9a> in <module>()
----> 1 with shelve.open("diz5") as db:
      2     with open("tmp1.txt") as instream:
      3         for line in instream:
      4             assert line.count("\t") == 1
      5             key, _tab, value = line.rstrip("\n").partition("\t")

AttributeError: DbfilenameShelf instance has no attribute '__exit__'

In [4]: 



I need to do intersection of dictionary key.
thanks  for the help
M.





More information about the Tutor mailing list