first things first: I love Duplicati, for me it’s the best free backup tool out there!
I have found a reproducable problem with GPG encryption and restoring in Duplicati - 2.0.8.1_beta_2024-05-07:
First, I create a backup with GPG encryption (public/private key pair)
To test restoring, I manually download all remote data and locally decrypt in my GPG client (my CyberSecurity guideline says, that I may never store the private key on my main PC, so I decrypt on a separate, offline PC)
Running a restore operation on the Duplicati UI give me the following error now:
2024-11-19 09:18:52 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-bda3cedb3a7684cea971c7190c2579ba1.dblock.zip (10,63 MB)
2024-11-19 09:18:52 +01 - [Verbose-Duplicati.Library.Main.BackendManager-AutomaticDecryptionDetection]: Filename extension "zip" does not match encryption module "gpg", guessing that it is not encrypted
2024-11-19 09:18:52 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-bda3cedb3a7684cea971c7190c2579ba1.dblock.zip (10,63 MB)
2024-11-19 09:18:52 +01 - [Retry-Duplicati.Library.Main.BackendManager-RetryGet]: Operation Get with file duplicati-bda3cedb3a7684cea971c7190c2579ba1.dblock.zip attempt 1 of 5 failed with message: The file duplicati-bda3cedb3a7684cea971c7190c2579ba1.dblock.zip was downloaded and had size 11144433 but the size was expected to be 11145989
System.Exception: The file duplicati-bda3cedb3a7684cea971c7190c2579ba1.dblock.zip was downloaded and had size 11144433 but the size was expected to be 11145989
bei Duplicati.Library.Main.BackendManager.DoGet(FileEntryItem item)
bei Duplicati.Library.Main.BackendManager.ThreadRun()
It seems that Duplicati correctly identified that these are not .zip.gpg but decrypted .zip files now. However, the file size after decryption is now smaller than expected and this causes errors. No files are restored.
I also tried the advanced parameters
–dont-read-manifests=true
–no-backend-verification=true
–no-encryption=true
but non of them could prevent the above exception.
I know that my data is not lost: I could always work on the GPG encrypted files from Duplicati directly. But this would be a pitty, as Duplicati is such a strong tool to increase the security to individually desired levels. Would it be possible to maybe add another advanced parameter like --dont-check-block-size=true?
Not sure how exactly you got into this situation, but Duplicati does not like when you change the remote files. Most often, a change here is an indication of some kind of error.
When you run the restore process, do you then use the “Restore from remote files” (i.e. no local configuration/database)? Or do you have a copy of the local database?
I am guessing here that the larger size is the encrypted size, and the smaller size is decrypted, but I do not understand how you got Duplicati to connect the two sizes for different filenames?
Very close! This option will skip the size checks (and hash checks):
--skip-file-hash-checks
Just for clarity: Duplicati has two configurable size parameters: block and remote volume size.
The size mismatch here is on the remote volume size (aka dblock-size), not the block size, which is 100KiB or 1MiB by default.
does the job. Regarding your question how I got into this situation: Yes, I like to test the full restore without local database, as if my PC got lost. So I manually download from cloud storage and manually decrypt all files. Then I run Duplicati restore in mode “Direct restore from backup files …” and Storage Type “Local folder or drive”.
I’m super happy that Duplicati recognizes that the .gpg extension is missing and is still able to connect the files. And yes, the bigger dblock file is the gpg encrypted one, the smaller is the decrypted zip file.
Though you helped increase my knowledge now , it would of course be much easier for other users if Duplicati would throw a warning instead of an exception in case it detects that the .zip.aes or .zip.gpg extensions changed to .zip and file sizes differ. I mean for users not setting the skip-file-hash-checks option.
I can see how that would work great for what you are doing.
However, for general use, the remote files should not change in any way.
A change is most likely a sign that something is not working.
If the flag is automatically set, it would be possible to replace an encrypted file with a malicious unencrypted file. In this case, Duplicati would continue without the user being aware of the change, and potentially restore incorrect data.
I prefer keeping the option to bypass checks as opt-in, so it is possible to work flexibly with the backups, but keep strict checks on by default.
, it would of course be much easier for other users if Duplicati would throw a warning instead of an exception in case it detects that the .zip.aes or .zip.gpg extensions changed to .zip and file sizes differ. I mean for users not setting the skip-file-hash-checks option.