[Tutor] Monitoring directories

[Mats Wichmann]

On 2/14/20 1:11 AM, Alan Gauld via Tutor wrote:
> On 14/02/2020 01:02, Nathan D'Elboux wrote:
> 
>> As this script is invoked currently via a cron job im not familiar
>> with what i would need to do to make this python script monitor in
>> daemon mode or how it would know  the newest files and the fact it
>> haunt already copied them over.
> 
> Store the copied filenames in a file perhaps?
> Either that or just store the time of the last run in a file. Then check
> the file creation date against the last run time and if newer copy the file.
> 

If you're going to stay with a cron job, the latter is the common
trick... you don't need to store anything in your timestamp file, just
"touch" it every time you've run and look only at files that have a
later modification time than the stamp file.

You may want to take a look at the underappreciated pathlib module if
you're going to be doing this kind of twiddling with paths - it has
support for a lot of useful things,like getting the stat of a file,
doing a touch, globbing, path joining, etc.  For example, off the top of
my head, probably riddled with errors:

from pathlib import Path

trg = Path("/home/target_dir")
src = Path("/home/source/")

stampfile = src / ".lastrun"
try:
    stamp = stampfile.stat().st_mtime
except FileNotFoundError:
    stamp = 0

# make a list of all the files in src, descending the tree:
filelist = [f for f in src.glob('**/*') if f.is_file()]

for f in filelist:
    if f.stat().st_mtime > stamp:
        # copy the file

stampfile.touch()



If you do want to go with a daemon, and use inotify, there's a PEP for
the daemon stuff (3143) and a sample implementation:

https://pypi.org/project/python-daemon/

Often it turns out you don't need a daemon.  But... it depends.