<div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman">Hi everyone,<?xml:namespace prefix = o ns = "urn:schemas-microsoft-com:office:office" /><o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman">Thanks for all suggestions. Let me just preface this by saying that I’m new to both python and programming. I started learning 3 months ago with online tutorials and reading the questions you guys post. So, thank you all very, very much…and I apologize if I’m doing something really stupid..:-)<SPAN style="mso-spacerun:
yes"> </SPAN>OK. I’ve solved the problem of opening
several files to process “as a batch” with glob.glob(). Only now did I realize that the program and files need to be in the same folder…. Now I have another problem. <o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman">1- I want to open several files and count the total number of words. If I do this with only 1 file, it works great. With several files ( now with glob), it outputs the total count for each file individually and not the whole corpus (see comment in the program below).<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman">2- I also want the program to output a word frequency list (we do this a lot in corpus linguistics). When I do this with only one file, the program works great (with a dictionary). With several files, I end up with several frequency lists, one for each file. This sounds
like a loop type of problem, doesn’t it? I looked at the indentations too and<SPAN style="mso-spacerun: yes"> </SPAN>I can’t find what the problem is. Your comments, suggestions, etc are greatly appreciated. Thanks again for all your help. Paulo<SPAN style="mso-spacerun: yes"> </SPAN><o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman">Here goes what I have.<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt; LINE-HEIGHT: 12pt; mso-line-height-rule: exactly"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"># The program is intended to output a word frequency list (including all words in all files) and the total word count <o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt; LINE-HEIGHT: 12pt; mso-line-height-rule: exactly"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman">def sortfile():<SPAN
style="mso-spacerun: yes"> </SPAN># I created a function<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN>filename = glob.glob('*.txt') # this works great! Thanks!<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN>for allfiles in filename:<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN>infile = open(allfiles, 'r')<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun:
yes"> </SPAN>lines = list(infile)<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN>infile.close()<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN>words = [] # initializes list of words<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN>wordcounter = 0<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun:
yes"> </SPAN>for line in lines: <o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN>line = line.lower() <SPAN style="mso-spacerun: yes"> </SPAN># after this, I have some clunky code to get rid of punctuation<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN>words = words + line.split() <o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN>wordfreq = [words.count(wrd)for wrd in
words] # counts the freq of each word in a list<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN>dictionary = dict(zip(words, wordfreq))<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN>frequency_list = [(dictionary[key], key)for key in dictionary] <o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN>frequency_list.sort()<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN
style="mso-spacerun: yes"> </SPAN>frequency_list.reverse()<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN><SPAN style="mso-spacerun: yes"> </SPAN>for item in frequency_list:<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN>wordcounter = wordcounter + 1<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN>print item<o:p></o:p></FONT></SPAN></div> <div class=MsoNormal
style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN>print "Total # of words:", wordcounter # <SPAN style="mso-spacerun: yes"> </SPAN>this will give the word count of the last file the program reads. <o:p></o:p></FONT></SPAN></div> <div class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman"><SPAN style="mso-spacerun: yes"> </SPAN><SPAN style="mso-spacerun: yes"> </SPAN>print "Total # of words:", wordcounter<SPAN style="mso-spacerun: yes"> </SPAN># if I put it here, I get the total count after each file<SPAN style="mso-spacerun: yes"> </SPAN><o:p></o:p></FONT></SPAN></div> <div
class=MsoNormal style="MARGIN: 0in 0in 0pt"><SPAN style="FONT-SIZE: 10pt"><FONT face="Times New Roman">sortfile() <o:p></o:p></FONT></SPAN></div> <div> </div>