np.genfromtxt StopIteration Error
I have been fighting with the genfromtxt function in numpy for a while now and am trying a slightly different approach. Here is the code:
import os import glob import numpy as np
fileList = [] filesList = []
for files in glob.glob("*.log"): fileName, fileExtension = os.path.splitext(files) fileList.append(fileName) filesList.append(files)
print('fileList = ', fileList) print('filesList = ', filesList)
fname = filesList print('fname = ', fname) data = np.genfromtxt(fname, usecols=(1), skip_header=27, skip_footer=1, encoding=None) print(data)
np.savetxt('fileList.dG', data, fmt='%12.9f', header='${d}') print(data.dG)
I am using the Spyder IDE which has a variable explorer which shows: filesList = ['C-VX3.log', '18-7.log', '14-7.log', '15-7.log'] fileList = ['C-VX3', '18-7', '14-7', '15-7'] so the lists that genfromtxt needs are being generated. Goggling 'numpy genfromtxt stopiteration error' does not seem to address this problem. At least, I didn't find plaything that I thought applied. I would greatly appreciate some assistance here. Thanks is advance. -- Stephen P. Molnar, Ph.D. www.molecular-modeling.net 614.312.7528 (c) Skype: smolnar1
I think genfromtxt() wants a filename as the first argument, and you have to tell it the entries in the file are strings not numerics. test.py ------------------------------------------ import os import glob import numpy as np fileList = [] filesList = [] for files in glob.glob("*.log"): fileName, fileExtension = os.path.splitext(files) fileList.append(fileName) filesList.append(files) print('fileList = ', fileList) print('filesList = ', filesList) fname = '/tmp/foo.txt' print('fname = ', fname) data = np.genfromtxt(fname, dtype=str) print(data) ------------------------------------------ Contents of /tmp/foo.txt ------------------------------------------ 15-7.log 18-7.log 14-7.log C-VX3.log ------------------------------------------ Sample run $ python --version Python 2.7.15+ $ python t.py ('fileList = ', ['15-7', '18-7', '14-7', 'C-VX3']) ('filesList = ', ['15-7.log', '18-7.log', '14-7.log', 'C-VX3.log']) ('fname = ', '/tmp/foo.txt') ['15-7.log' '18-7.log' '14-7.log' 'C-VX3.log'] Is that any help? On Fri, Oct 11, 2019 at 12:41 PM Stephen P. Molnar <s.molnar@sbcglobal.net> wrote:
I have been fighting with the genfromtxt function in numpy for a while now and am trying a slightly different approach.
Here is the code:
import os import glob import numpy as np
fileList = [] filesList = []
for files in glob.glob("*.log"): ?????? fileName, fileExtension = os.path.splitext(files) ?????? fileList.append(fileName) ?????? filesList.append(files)
print('fileList = ', fileList) print('filesList = ', filesList)
fname = filesList print('fname = ', fname) data = np.genfromtxt(fname, usecols=(1), skip_header=27, skip_footer=1, encoding=None) print(data)
np.savetxt('fileList.dG', data, fmt='%12.9f', header='${d}') print(data.dG)
I am using the Spyder IDE which has a variable explorer which shows:
filesList = ['C-VX3.log', '18-7.log', '14-7.log', '15-7.log'] fileList = ['C-VX3', '18-7', '14-7', '15-7']
so the lists that genfromtxt needs are being generated.
Goggling 'numpy genfromtxt stopiteration error' does not seem to address this problem. At least, I didn't find plaything that I thought applied.
I would greatly appreciate some assistance here.
Thanks is advance.
-- Stephen P. Molnar, Ph.D. www.molecular-modeling.net 614.312.7528 (c) Skype: smolnar1
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Thanks for the reply. Keep in mind that i am a Chemist, not an IT person. I used to be a marginally proficient FORTRAN II user in the ancient past. I tried running your code. Please see my comments/questing below: On 10/11/2019 01:12 PM, Bennet Fauber wrote:
I think genfromtxt() wants a filename as the first argument, and you have to tell it the entries in the file are strings not numerics.
test.py ------------------------------------------ import os import glob import numpy as np
fileList = [] filesList = []
for files in glob.glob("*.log"): fileName, fileExtension = os.path.splitext(files) fileList.append(fileName) filesList.append(files)
print('fileList = ', fileList) print('filesList = ', filesList
fname = '/tmp/foo.txt' There is no '/temp/foo.txt' Where did it come from in your example? print('fname = ', fname) data = np.genfromtxt(fname, dtype=str) print(data) ------------------------------------------
Contents of /tmp/foo.txt ------------------------------------------ 15-7.log 18-7.log 14-7.log C-VX3.log ------------------------------------------
Sample run I'm using python 3.7.3, should this make a difference?
$ python --version Python 2.7.15+
$ python t.py ('fileList = ', ['15-7', '18-7', '14-7', 'C-VX3']) ('filesList = ', ['15-7.log', '18-7.log', '14-7.log', 'C-VX3.log']) ('fname = ', '/tmp/foo.txt') ['15-7.log' '18-7.log' '14-7.log' 'C-VX3.log']
Is that any help? if I use data = np.genfromtxt('14-7.log', dtype=str, usecols=(1), skip_header=27, skip_footer=1, encoding=None) with a specific file name. in this example 14-7, I get the resutt I desired:
# 14-7 -9.960902669 -8.979504781 -8.942611364 -8.915523010 -8.736508831 -8.663387139 -8.410739711 -8.389146347 -8.296798909 -8.168454106 -8.127990818 -8.127103774 -7.979090739 -7.941872682 -7.900766215 -7.881485228 -7.837826485 -7.815909505 -7.722540286 -7.720346742 so, my question is; why the StopIteration error message in my original query? Why is the dcrtipt not iterating over the log files? Sorry to be so dense.
On Fri, Oct 11, 2019 at 12:41 PM Stephen P. Molnar <s.molnar@sbcglobal.net> wrote:
I have been fighting with the genfromtxt function in numpy for a while now and am trying a slightly different approach.
Here is the code:
import os import glob import numpy as np
fileList = [] filesList = []
for files in glob.glob("*.log"): ?????? fileName, fileExtension = os.path.splitext(files) ?????? fileList.append(fileName) ?????? filesList.append(files)
print('fileList = ', fileList) print('filesList = ', filesList)
fname = filesList print('fname = ', fname) data = np.genfromtxt(fname, usecols=(1), skip_header=27, skip_footer=1, encoding=None) print(data)
np.savetxt('fileList.dG', data, fmt='%12.9f', header='${d}') print(data.dG)
I am using the Spyder IDE which has a variable explorer which shows:
filesList = ['C-VX3.log', '18-7.log', '14-7.log', '15-7.log'] fileList = ['C-VX3', '18-7', '14-7', '15-7']
so the lists that genfromtxt needs are being generated.
Goggling 'numpy genfromtxt stopiteration error' does not seem to address this problem. At least, I didn't find plaything that I thought applied.
I would greatly appreciate some assistance here.
Thanks is advance.
-- Stephen P. Molnar, Ph.D. www.molecular-modeling.net 614.312.7528 (c) Skype: smolnar1
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
-- Stephen P. Molnar, Ph.D. www.molecular-modeling.net 614.312.7528 (c) Skype: smolnar1
Hi Stephen, Is this not what your original question to this list was about? See https://mail.python.org/pipermail/numpy-discussion/2019-October/080130.html and replies. I still believe that you _can't_ give genfromtxt file names in an iterable. The iterable input is only inteded to contain the contents of a single file, probably as if read using `f.readlines()`. Genfromtxt seems to read data from at most one file with each call. Regards, András On Fri, Oct 11, 2019 at 8:41 PM Stephen P. Molnar <s.molnar@sbcglobal.net> wrote:
Thanks for the reply.
Keep in mind that i am a Chemist, not an IT person. I used to be a marginally proficient FORTRAN II user in the ancient past.
I tried running your code. Please see my comments/questing below:
On 10/11/2019 01:12 PM, Bennet Fauber wrote:
I think genfromtxt() wants a filename as the first argument, and you have to tell it the entries in the file are strings not numerics.
test.py ------------------------------------------ import os import glob import numpy as np
fileList = [] filesList = []
for files in glob.glob("*.log"): fileName, fileExtension = os.path.splitext(files) fileList.append(fileName) filesList.append(files)
print('fileList = ', fileList) print('filesList = ', filesList
fname = '/tmp/foo.txt' There is no '/temp/foo.txt' Where did it come from in your example? print('fname = ', fname) data = np.genfromtxt(fname, dtype=str) print(data) ------------------------------------------
Contents of /tmp/foo.txt ------------------------------------------ 15-7.log 18-7.log 14-7.log C-VX3.log ------------------------------------------
Sample run I'm using python 3.7.3, should this make a difference?
$ python --version Python 2.7.15+
$ python t.py ('fileList = ', ['15-7', '18-7', '14-7', 'C-VX3']) ('filesList = ', ['15-7.log', '18-7.log', '14-7.log', 'C-VX3.log']) ('fname = ', '/tmp/foo.txt') ['15-7.log' '18-7.log' '14-7.log' 'C-VX3.log']
Is that any help? if I use data = np.genfromtxt('14-7.log', dtype=str, usecols=(1), skip_header=27, skip_footer=1, encoding=None) with a specific file name. in this example 14-7, I get the resutt I desired:
# 14-7 -9.960902669 -8.979504781 -8.942611364 -8.915523010 -8.736508831 -8.663387139 -8.410739711 -8.389146347 -8.296798909 -8.168454106 -8.127990818 -8.127103774 -7.979090739 -7.941872682 -7.900766215 -7.881485228 -7.837826485 -7.815909505 -7.722540286 -7.720346742
so, my question is; why the StopIteration error message in my original query? Why is the dcrtipt not iterating over the log files?
Sorry to be so dense.
On Fri, Oct 11, 2019 at 12:41 PM Stephen P. Molnar <s.molnar@sbcglobal.net> wrote:
I have been fighting with the genfromtxt function in numpy for a while now and am trying a slightly different approach.
Here is the code:
import os import glob import numpy as np
fileList = [] filesList = []
for files in glob.glob("*.log"): ?????? fileName, fileExtension = os.path.splitext(files) ?????? fileList.append(fileName) ?????? filesList.append(files)
print('fileList = ', fileList) print('filesList = ', filesList)
fname = filesList print('fname = ', fname) data = np.genfromtxt(fname, usecols=(1), skip_header=27, skip_footer=1, encoding=None) print(data)
np.savetxt('fileList.dG', data, fmt='%12.9f', header='${d}') print(data.dG)
I am using the Spyder IDE which has a variable explorer which shows:
filesList = ['C-VX3.log', '18-7.log', '14-7.log', '15-7.log'] fileList = ['C-VX3', '18-7', '14-7', '15-7']
so the lists that genfromtxt needs are being generated.
Goggling 'numpy genfromtxt stopiteration error' does not seem to address this problem. At least, I didn't find plaything that I thought applied.
I would greatly appreciate some assistance here.
Thanks is advance.
-- Stephen P. Molnar, Ph.D. www.molecular-modeling.net 614.312.7528 (c) Skype: smolnar1
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
-- Stephen P. Molnar, Ph.D. www.molecular-modeling.net 614.312.7528 (c) Skype: smolnar1
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
What happens if you use this? for txtfile in filesList: data = np.genfromtxt(txtfile, dtype=str, usecols=(1), skip_header=27, skip_footer=1, encoding=None) print(data) That pulls one file name from filesList, stuffs the name into txtfile, which is then provided to genfromtxt(). The point is that genfromtxt() wants one thing; you have to give it one thing. The above gives it one thing, but runs it for however many things are in the filesList. Does that help? On Fri, Oct 11, 2019 at 2:41 PM Stephen P. Molnar <s.molnar@sbcglobal.net> wrote:
Thanks for the reply.
Keep in mind that i am a Chemist, not an IT person. I used to be a marginally proficient FORTRAN II user in the ancient past.
I tried running your code. Please see my comments/questing below:
On 10/11/2019 01:12 PM, Bennet Fauber wrote:
I think genfromtxt() wants a filename as the first argument, and you have to tell it the entries in the file are strings not numerics.
test.py ------------------------------------------ import os import glob import numpy as np
fileList = [] filesList = []
for files in glob.glob("*.log"): fileName, fileExtension = os.path.splitext(files) fileList.append(fileName) filesList.append(files)
print('fileList = ', fileList) print('filesList = ', filesList
fname = '/tmp/foo.txt' There is no '/temp/foo.txt' Where did it come from in your example? print('fname = ', fname) data = np.genfromtxt(fname, dtype=str) print(data) ------------------------------------------
Contents of /tmp/foo.txt ------------------------------------------ 15-7.log 18-7.log 14-7.log C-VX3.log ------------------------------------------
Sample run I'm using python 3.7.3, should this make a difference?
$ python --version Python 2.7.15+
$ python t.py ('fileList = ', ['15-7', '18-7', '14-7', 'C-VX3']) ('filesList = ', ['15-7.log', '18-7.log', '14-7.log', 'C-VX3.log']) ('fname = ', '/tmp/foo.txt') ['15-7.log' '18-7.log' '14-7.log' 'C-VX3.log']
Is that any help? if I use data = np.genfromtxt('14-7.log', dtype=str, usecols=(1), skip_header=27, skip_footer=1, encoding=None) with a specific file name. in this example 14-7, I get the resutt I desired:
# 14-7 -9.960902669 -8.979504781 -8.942611364 -8.915523010 -8.736508831 -8.663387139 -8.410739711 -8.389146347 -8.296798909 -8.168454106 -8.127990818 -8.127103774 -7.979090739 -7.941872682 -7.900766215 -7.881485228 -7.837826485 -7.815909505 -7.722540286 -7.720346742
so, my question is; why the StopIteration error message in my original query? Why is the dcrtipt not iterating over the log files?
Sorry to be so dense.
On Fri, Oct 11, 2019 at 12:41 PM Stephen P. Molnar <s.molnar@sbcglobal.net> wrote:
I have been fighting with the genfromtxt function in numpy for a while now and am trying a slightly different approach.
Here is the code:
import os import glob import numpy as np
fileList = [] filesList = []
for files in glob.glob("*.log"): ?????? fileName, fileExtension = os.path.splitext(files) ?????? fileList.append(fileName) ?????? filesList.append(files)
print('fileList = ', fileList) print('filesList = ', filesList)
fname = filesList print('fname = ', fname) data = np.genfromtxt(fname, usecols=(1), skip_header=27, skip_footer=1, encoding=None) print(data)
np.savetxt('fileList.dG', data, fmt='%12.9f', header='${d}') print(data.dG)
I am using the Spyder IDE which has a variable explorer which shows:
filesList = ['C-VX3.log', '18-7.log', '14-7.log', '15-7.log'] fileList = ['C-VX3', '18-7', '14-7', '15-7']
so the lists that genfromtxt needs are being generated.
Goggling 'numpy genfromtxt stopiteration error' does not seem to address this problem. At least, I didn't find plaything that I thought applied.
I would greatly appreciate some assistance here.
Thanks is advance.
-- Stephen P. Molnar, Ph.D. www.molecular-modeling.net 614.312.7528 (c) Skype: smolnar1
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
-- Stephen P. Molnar, Ph.D. www.molecular-modeling.net 614.312.7528 (c) Skype: smolnar1
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
participants (3)
-
Andras Deak -
Bennet Fauber -
Stephen P. Molnar