I want to run a web-server which displays information about files on my filesystems. The info will be stored in either a mysql or sqlite database. The number of files to be scanned can be many and huge in size. I want to click on a link which goes to a specified page/resource on the server which starts the filescanning, generates checksums for the files processed ( sha/md5 ) and stores this info in the database. The process cannot block the server, at least not the filescanning itself. The database interactions is not that critical. It would be nice to have some kind of indication available for users to see that a filescan was in progress, perhaps even some kind of progress indication.
Can anybody show me how to do a non-blocking filescan like I've described in a Twisted-based web-application which generates sha/md5-checkums in the process?