[Tutor] Question for Strategy for Directory Monitor

[Saran Ahluwalia]

Good Evening :

Here is what I want me program to do:

• *Monitor* a folder for files that are dropped throughout the day

• When a file is dropped in the folder the program should scan the file

o IF all the contents in the file have the same length (let's assume line
length)

o THEN the file should be moved to a "success" folder and a text file
written indicating the total number of records/lines/words processed

o IF the file is empty OR the contents are not all of the same length

o THEN the file should be moved to a "failure" folder and a text file
written indicating the cause for failure (for example: Empty file or line
100 was not the same length as the rest).

I want to thank Martin Brown for his guidance and feedback. I welcome any
and all feedback on the following

import os
import time
import glob
import sys

def initialize_logger(output_dir):
    logger = logging.getLogger()
    logger.setLevel(logging.DEBUG)

    # create console handler and set level to info
    handler = logging.StreamHandler()
    handler.setLevel(logging.INFO)
    formatter = logging.Formatter("%(levelname)s - %(message)s")
    handler.setFormatter(formatter)
    logger.addHandler(handler)

    # create error file handler and set level to error
    handler = logging.FileHandler(os.path.join(output_dir,
"error.log"),"w", encoding=None, delay="true")
    handler.setLevel(logging.ERROR)
    formatter = logging.Formatter("%(levelname)s - %(message)s")
    handler.setFormatter(formatter)
    logger.addHandler(handler)

    # create debug file handler and set level to debug
    handler = logging.FileHandler(os.path.join(output_dir, "all.log"),"w")
    handler.setLevel(logging.DEBUG)
    formatter = logging.Formatter("%(levelname)s - %(message)s")
    handler.setFormatter(formatter)
    logger.addHandler(handler)


def main(dirslist):
    while True:
        for file in os.listdir(dirslist) :
        return validate_files(file)
        time.sleep(5)

if __name__ == "__main__":
    main()


*#Helper Functions for the Success and Failure Folder Outcomes,
respectively*

    def file_len(filename):
        with open(filename) as f:
            for i, l in enumerate(f):
                pass
            return i + 1

    def copyFile(src, dest):
        try:
            shutil.copy(src, dest)
        # eg. src and dest are the same file
        except shutil.Error as e:
            print('Error: %s' % e)
        # eg. source or destination doesn't exist
        except IOError as e:
            print('Error: %s' % e.strerror)

def move_to_failure_folder_and_return_error_file():
    os.mkdir('Failure')
    copyFile(filename, 'Failure')
    initialize_logger('rootdir/Failure')
    logging.error("Either this file is empty or the lines")


def move_to_success_folder_and_read(file):
    os.mkdir('Success')
    copyFile(filename, 'Success')
    print("Success", file)
    return file_len()

#This simply checks the file information by name
def fileinfo(file):
    filename = os.path.basename(file)
    rootdir = os.path.dirname(file)
    lastmod = time.ctime(os.path.getmtime(file))
    creation = time.ctime(os.path.getctime(file))
    filesize = os.path.getsize(file)
    return filename, rootdir, lastmod, creation, filesize

if __name__ == '__main__':
   import sys
   validate_files(sys.argv[1:])

I am trying to specifically address the fact that the program does not:

   - Assumes that all filenames come directly from the commandline.  No
   searching of a directory.


   - The present code does not move any files to success or failure
directories (I
   have added functions at the end that could serve as substitutes)


   - The present code doesn't calculate or write to a text file.


   - The present code runs once through the names, and terminates.  It doesn't
   "monitor" anything  - I think that I have added the correct while loop.


   - The present code doesn't check for zero-length files

I have attempted to address these but, as a novice, am not sure what is
best practice.

Sincerely,

Saran