Stefanie,

 

Good questions.

 

 

Here is how to create an in-memory kernel from a numpy array:

 

Example

>>> from pysnptools.kernelreader import KernelData

>>> kerneldata = KernelData(iid=[['fam0','iid0'],['fam0','iid1']], val=[[1.,.5],[.5,1.]])

>>> print((kerneldata.val[0,1], kerneldata.iid_count))

(0.5, 2)

 

This kerneldata object can be passed in to single_snp.

 

You can also write that kerneldata object to *.kernel.npz format like this:

from pysnptools.kernelreader import KernelNpz
KernelNpz.write("tempdir/toydata.kernel.npz",kerneldata)

(Aside: *.kernel.npz is a version of *.npz that includes not only the kernel itself, but also information about the individual id’s and optimal metadata)

 

After that you could pass

KernelNpz("tempdir/toydata.kernel.npz")

to single_snp

 

For more details, follow up with me or see https://fastlmm.github.io/PySnpTools/#module-pysnptools.kernelreader

 

 

If K0 is not provided, it is computed from the test SNP data. (Aside: when leave-one-chromsome-out is applicable, there is actually one K0 for each chromosome left out). The SNP data is standardized (made to have mean 0 and std dev 1 and missing filled with 0) to Gx and then K0 is Gx.T.dot(Gx) where T is numpy’s transpose and dot is (for some reason) numpy’s matrix multiply.

 

Let me know if this is clear. I’m happy to help with Python format conversion problems if you let me know about the format you’re starting with.

 

Thanks again for using FaST-LMM!

 

 

 

From: Stefanie Lück <luecks@gmail.com>
Sent: Monday, May 10, 2021 12:25 AM
To: fastlmm-user@python.org
Subject: [fastlmm-user] Kinship
Importance: High

 

Hi again,

 

I have two questions using the similarity matrix in single_snp:

 

1) Which format do I have to provide if I use the npz format? My npz matrix throws the error below.

 

2) If I don't provide K0, how is the similarity matrix calculated and is it possible to store the matrix for other runs?

 

Thanks a lot

Stefanie

 

Error:

Traceback (most recent call last):
  File "C:/Users//PycharmProjects/BCC_Experiments/lmm/lmm01.py", line 27, in <module>
    results_df = single_snp(bed_fn, pheno_fn, K0=k)
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\fastlmm\association\single_snp.py", line 246, in single_snp
    runner = runner)
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\pysnptools\util\mapreduce1\mapreduce.py", line 202, in map_reduce
    result = runner.run(dist)
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\pysnptools\util\mapreduce1\runner\local.py", line 48, in run
    result = _run_all_in_memory(distributable)
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\pysnptools\util\mapreduce1\runner\__init__.py", line 30, in _run_all_in_memory
    return work.reduce(result_sequence)
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\pysnptools\util\mapreduce1\mapreduce.py", line 77, in reduce
    return self.reducer(output_seq)
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\fastlmm\association\single_snp.py", line 228, in reducer_closure
    frame = pd.concat(frame_sequence)
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\pandas\core\reshape\concat.py", line 295, in concat
    sort=sort,
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\pandas\core\reshape\concat.py", line 339, in __init__
    objs = list(objs)
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\pysnptools\util\mapreduce1\runner\__init__.py", line 14, in work_sequence_to_result_sequence
    result = work()
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\pysnptools\util\mapreduce1\mapreduce.py", line 65, in <lambda>
    yield lambda i=i, input_arg=input_arg: self.dowork(i, input_arg)  # the 'i=i',etc is need to get around a strangeness in Python
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\pysnptools\util\mapreduce1\mapreduce.py", line 92, in dowork
    result = _run_all_in_memory(work)
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\pysnptools\util\mapreduce1\runner\__init__.py", line 25, in _run_all_in_memory
    return work()
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\pysnptools\util\mapreduce1\mapreduce.py", line 91, in <lambda>
    work = lambda : self.mapper(input_arg)
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\fastlmm\association\single_snp.py", line 211, in nested_closure
    K0_chrom = _K_per_chrom(K0 or G0 or test_snps, chrom, test_snps.iid)
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\fastlmm\association\single_snp.py", line 301, in _K_per_chrom
    return SnpKernel(K_all.snpreader[:,K_all.pos[:,0] != chrom],K_all.standardizer)
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\pysnptools\kernelreader\snpkernel.py", line 150, in pos
    return self.snpreader.pos
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\pysnptools\snpreader\snpreader.py", line 404, in pos
    return self.col_property
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\pysnptools\pstreader\pstnpz.py", line 67, in col_property
    self._run_once()
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\pysnptools\pstreader\pstnpz.py", line 82, in _run_once
    self._row = data['row']
  File "C:\Users\\AppData\Local\Continuum\anaconda3\envs\gwas_flow\lib\site-packages\numpy\lib\npyio.py", line 259, in __getitem__
    raise KeyError("%s is not a file in the archive" % key)
KeyError: 'row is not a file in the archive'

Process finished with exit code 1