Hi,
TLDR; Local and remote backup went fine for months. After some connection issues and reboots of both the source and Duplicati machines, source shows 0 bytes even though the directory is there and readable. Is there a way I can force Duplicati to recheck the source directory and correctly index the files there?
Detailed version:
I’ve been running Duplicati for a client for about 3 months now, so far without a problem. There are two jobs, one for a local backup, one for a remote backup to a server of mine via SSH.
The system that runs Duplicati and is the local target for the backup is a Raspberry Pi 4 with Raspberry Pi OS (bullseye, 64-Bit) and an internal 1 TB SATA SSD. The local source is a Synology NAS. The remote target is an Unraid server that runs an SSH server as a Docker container, exclusive to that one backup job.
So far I had the local backup set for 2 AM and the remote backup for 3 AM, which worked fine.
Up to now, the data on the NAS had been about 64-70 GB, which after the initial full backup, took about 7-10 minutes to make incremental backups to both targets, since it’s a small office operation and they don’t generate that much new data on a daily basis. Recently, the client swapped out his main computer and unbeknownst to me, backed up about 180 GB of additional data from the old computer to the NAS.
This led to the local backup job to run for about 4.5 hours the next night, pushing the remote backup closer to around 7 AM - which would have been fine, except my Unraid server performs its own backup operations for the Docker configurations at 7 AM, and stops all Docker containers to do so, leading to the server going offline at 7 AM.
This led to some sort of problem with the remote backup job: Unexpected difference in fileset version nn: DD/MM/YYYY 03:00:00 (database id: 30), found 136843 entries, but expected 136844. I managed to fix this by deleteing a number of filesets until the error went away.
I went to run the remote backup to my server manually in the morning, expecting it to take a few hours to backup 180 GB over a DSL line. I checked later that evening and found that the backup job was stuck at some point while reading files. As I only had remote access to the Raspberry Pi, I went to reboot it, which led to some sort of issue with the shutdown. I then asked the client to power cycle it, during which it looks like he also accidentally shut down the NAS.
I went over there today, restarted the NAS and went to perform a local backup first, when I noticed that the Duplicati GUI shows the source for both backup jobs to be 0 bytes. When checking on the command line level, the source directory is there and readable (it is being mounted on the Raspberry on boot via an entry in /etc/fstab). I can see and also select it and its subdirectories when editing the jobs in the GUI or creating a new job.
Is there a way I can force Duplicati to recheck the source directory and correctly index the files there?