Real-time backup

Sounds awesome!

The implementation is based on the two options --changed-files and --deleted-files options. Using the commandline, you can set these two to a path separated list (using ; on Windows and : on others). Once they are set, the file scanner is not used, and it only scans the --changed-files and removes the --deleted-files.

If you are using USN, it should be possible to store the previous USN time-stamp and then query USN to give the change lists. This will work even if you restart Duplicati in the meantime.

For the FileWatcher approach, you have the problem that it is not guaranteed to keep running, so you need to handle the case where it is started (and thus cannot help you and must run a full scan). If it has been running since the last backup, you can use the list and avoid the scan.

The list may be really large (many files are changed), so perhaps you need something like a database to store the filenames. It is also possible that we need to re-design the way we pass the list of filenames to avoid storing them in-memory( i.e. using a file or similar).

I am not sure how to do this the best, but perhaps it would make sense to start the FileWatcher when running a backup. This first run will then use the scan, but keep the FileWatcher running, such that the next backup is using the data from the FileWatcher.

If this is the “right way”, I think you can inject this into the “Runner.cs” class in the Duplicati.Server project:

Let me know if you have any questions.

1 Like