Duplicati Running in Container on Debian | Backing up to Mounted CIFS

[wingman1487]

I don’t know if this is just a weird setup, but for whatever reason when using the LinuxServer build of Duplicati (Docker Hub) I am plagued with size check issues, not sure if it’s just not how it’s supposed to be used or if I’ve overly complicated the install, but this seems like a pretty easy setup.

Error Sample:

• 2022-03-22 22:32:43 +00 - [Warning-Duplicati.Library.Main.Operation.FilelistProcessor-MissingRemoteHash]: remote file duplicati-b644ea6d6f91a4cfbb5a8b9ff7fdd81af.dblock.zip.aes is listed as Verified with size 247463936 but should be 262133629, please verify the sha256 hash “wtrZfDMz6AAN6pPupd5qaXszuMiFk5ojEF7r0Z/PDsQ=”

Setup:

• Host: Physical
• Model: Dell R620
• OS: Debian 10 Buster
• Docker: Version 20.10.13, Build a224086
• Network Shares: CIFS

What I’ve done here is setup this physical host as an “in between” for all my network shares, some on local Windows hosts, USB mounted HDD to Router, and external NAS box connected in via VPN. I mount all the shares as CIFS due to the majority of my Source data coming in from Windows and going back out to Windows (plus others but they are all CIFS anyways).

Now I’m not sure why, maybe it’s the OS mounting CIFS as I’ve read some folks saying that Duplicati doesn’t like things like CIFS caching being enable, other issues I’ve seen are with mono version (in the container the mono version is 5.20.1.34). I really don’t know, but every backup I setup and run has these Warnings all over the logs, and they are correct, the size matches the “Verified size” and not the “should be” size, is there something with this setup that Duplicati doesn’t like?

Windows > Share > Debian > Mount (CIFS) > Docker > Mount (Volume) > Duplicati > Source > Windows Share (all mounted the same way)

[ts678]

@wingman1487

size 247463936 but should be 262133629

Typical symptom. The file size got truncated to hexadecimal EC00000, suggesting FS or SMB goofed.

Windows efforts to use Process Monitor to inspect the writes (and any errors) failed. Tool did not work.
Linux has strace which can do similar things, but AFAIK nobody has attempted it on affected system.

Gostev’s digest from 16.12.2019 - SMB share repository on a cheap NAS talks about SMB challenges.
When to use SMB WriteThrough in Windows Server 2019 talks about Windows /WRITETHROUGH option which seemingly is similar to the Linux directio mount option seen on the mount.cifs manual page.

Care to give any of the above a try, whether it’s debugging, seeking a workaround, or some other idea?

@wingman1487

has these Warnings all over the logs

If they happen easily enough, that makes troubleshooting easier. If you can get into the Docker and run Duplicati.CommandLine.BackendTester.exe using URL from an Export As Command-line edited to use empty test folder, it might even be possible to see some errors that way. There’s also a manual test tool.

@MrWhite

I was able to reproduce the issue by unplugging/plugging the network cable during backup. Sometimes this does not cause an issue, sometimes the file size is different, and sometimes the files does not get created at all. There are likely many ways this issue could occur but proving that something as simple as a network interruption can corrupt a backup set is unacceptable.

I am working on adding file size validation to the FileBackend module code that would detect and retry the file creation if the file is missing or the wrong size.

is a future hope to increase robustness of less-than-perfect file share behavior by testing and retrying.