how to solve memory

meInvent bbird jobmattcon at gmail.com
Sat Jun 11 01:55:54 EDT 2016


i use sage cloud to run, it killed my program

i checked total memory is large, how much memory it need, 8GB*10, 80GB?

top - 05:55:16 up 10:05,  1 user,  load average: 0.35, 0.63, 0.56
Tasks:  13 total,   1 running,  12 sleeping,   0 stopped,   0 zombie
%Cpu(s):  0.5 us,  0.4 sy,  0.0 ni, 99.1 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:  26755056 total,  6317696 used, 20437360 free,   611836 buffers
KiB Swap: 67108860 total,   468688 used, 66640172 free.  1223512 cached Mem

def DFS(b, deep, maxx, sourceoperators, path): 
    initlist = [] 
    if deep > 0: 
        print("deep=", deep) 
        for aa,bb in itertools.combinations(sourceoperators, 2): 
            print(aa,bb) 
            if deep == maxx: 
                finalresult = [] 
                op1xy = [aa[b[i][0:1]+b[i][1:2]] for i in range(len(b))]
                op1yz = [aa[b[i][1:2]+b[i][2:3]] for i in range(len(b))]
                op1xz = [aa[b[i][0:1]+b[i][2:3]] for i in range(len(b))]
                op2xy = [bb[b[i][0:1]+b[i][1:2]] for i in range(len(b))]
                op2yz = [bb[b[i][1:2]+b[i][2:3]] for i in range(len(b))]
                op2xz = [bb[b[i][0:1]+b[i][2:3]] for i in range(len(b))]
                if sum(op1xy) == 54: 
                    path.append([(deep, aa, b, "xy")]) 
                else:
                    initlist.append(op1xy)
                if sum(op1yz) == 54: 
                    path.append([(deep, aa, b, "yz")]) 
                else:
                    initlist.append(op1yz)
                if sum(op1xz) == 54: 
                    path.append([(deep, aa, b, "xz")]) 
                else:
                    initlist.append(op1xz)
                if sum(op2xy) == 54: 
                    path.append([(deep, bb, b, "xy")]) 
                else:
                    initlist.append(op2xy)
                if sum(op2yz) == 54: 
                    path.append([(deep, bb, b, "yz")]) 
                else:
                    initlist.append(op2yz)
                if sum(op2xz) == 54: 
                    path.append([(deep, bb, b, "xz")]) 
                else:
                    initlist.append(op2xz)
            else: 
                level = [] 
                for j in range(len(b)): 
                    op1xy = [aa[b[j][i]] for i in range(len(b[j]))] 
                    op2xy = [bb[b[j][i]] for i in range(len(b[j]))] 
                    if sum(op1xy) == 54: 
                        path.append([(deep, aa, b[j], "xy")]) 
                    else:
                        initlist.append(op1xy)
                    if sum(op2xy) == 54: 
                        path.append([(deep, bb, b[j], "xy")]) 
                    else:
                        initlist.append(op2xy)
                    level.extend([op1xy, op2xy]) 
    if deep == maxx: 
        b = [] 
    print("initlist=")
    print(len(initlist))
    for aaa,bbb in itertools.combinations(initlist, 2): 
        b.append([str(i)+str(j) for i,j in zip(aaa, bbb)]) 
    if deep > 0:
        path2 = DFS(b, deep-1, maxx, sourceoperators, path) 
        path.append(path2)
    return path 

path = [] 
mresult = DFS(b, 2, 2, mylist, path) 




On Saturday, June 11, 2016 at 8:34:16 AM UTC+8, MRAB wrote:
> On 2016-06-11 00:31, meInvent bbird wrote:
> > it is quite ridiculous,
> > this time i am sure that i put correct indentation
> > and add a else statement to make sure recursive call inside the
> > if statement , it still memory error,
> >
> > where is the memory error?
> >
> > is itertools.combinations so big for the list?
> >
> [snip]
> 
> How long is initlist?
> 
> When I ran the code, it said over 100_000 items.
> 
> How many combinations would there be?
> 
> Over 10_000_000_000.
> 
> That's how long the list 'b' would be.
> 
> You'll need 10s of gigabytes of memory and a lot of patience!



More information about the Python-list mailing list