[Tutor] Need help appending data to a logfile

[Alan Gauld]

On 30/06/13 04:41, Matt D wrote:

> So i have a program that gets pickled data from a thread and displays
> the data (9 strings) in TextCtrl fields in the UI.

The fact that it is a UI is largely irrelevant apart from the
fact that it forces an event driven architecture.

>  When the pickle is
> received it is processed like this:
>
> def display_data(self,event):
...
>      attrs = pickle.loads(pickled_dict)
>      self.update(attrs)
>
> attrs is passed into the update function:
>
>   def update(self, field_values):
>      #  logger code----------
>      self.logfile.write('\n')
>      self.logfile.write('%s,'%(str(strftime("%Y-%m-%d %H:%M:%S",
> localtime()))))
>      for k,v in self.fields.items():
>          f = field_values.get(k, None)
>          if f:
>              self.logfile.write('%s,'%(str(f)))
>       #end logger code ----------------

Why keep the logfile open? Why not store the logfile name
(as selected by the user) and open the file in the update
function and close it after writing?

> so i have the logfile.txt formatted in a was that is very useful for
> later inspection with time series analysis.

> ...  up near the main class constructor the log file is opened like
> this:
>
>      self.logfile = open('logfile.txt', 'a')

This is the bit I don't understand. Is there a reason you are
using a hard coded filename? That is the source of much of
your complexity. And why are you opening it so early? Why
not wait till you need to write?

> it is stored in the home folder

That's a choice you make with the hard coded name. If you
allow the user to select the name (and folder) it can
be stored anywhere. The user choice can be via a GUI
or via a config file or via a startup argument.

> and gets appended every time the program
> is run.  Only problem is i need multiple log files

Why do you need multiple log files?
Is there one per user and you have multiple users?
Or do you need multiple files per user?
Or do you only "need" multiple files because you hardcoded
the name and then allow the user to choose their own name?

> and the user needs to
> be able to choose the file from the UI.

Are you saying the user needs to be able to rename
the file in the middle of processing? If so thats
an unusual requirement but there are ways of doing
it that are easier than what you seem to be doing.

>      def openFile(self, evt):
>          with wx.FileDialog(self, "Choose a file", os.getcwd(), "",
> "*.txt*", wx.OPEN) as dlg:
>             if dlg.ShowModal() == wx.ID_OK:
>                  path = dlg.GetPath()
>                  mypath = os.path.basename(path)
>                  uf = open(mypath, "a")
>                  uf.write(self.logfile)

And as we've pointed out you can't write the logfile
*object* you need to read the data and then write it out.
And to read it you need to mess with the cursor using seek()
But if you have already closed the file due to the way
you opened it with 'with' you won;t be able to seek or write.
Hence, I suspect, the error message.

> I have been unable to get the logfile.txt written
 > into the user opened file.

But I'd really like to understand why you believe this
is needed. Why not just get the user to specify the file
up front and then do all the logging into that.


PS. In the update function you have this:

 >      self.logfile.write('\n')
 >      self.logfile.write('%s,'%(str(strftime("%Y-%m-%d %H:%M:%S",
 > localtime()))))

It might help if you separate out the string formatting from the file 
writing

output = ('\n')
output += ('%s,'%(str(strftime("%Y-%m-%d %H:%M:%S",localtime()))))
self.logfile.write(output)

And hopefully this makes it obvious that the formatting is overly 
complex you don't need all the string conversion stuff. It can
just be:

)
output = '\n%s,' % strftime("%Y-%m-%d %H:%M:%S",localtime())
self.logfile.write(output)



HTH
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/