I’ve been using duplicati for small to medium-sized backups (up to 100GB or so) for several years, and have been extremely impressed overall. I recently added a much larger (~3TB) backup to be done over ssh/sftp using AES and otherwise standard settings. Just before the initial backup completed (which took several days), it returned a series of failure messages:
Fatal error
Duplicati.Library.Interface.UserInformationException: Found 1 files that are missing from the remote storage, please run repair
at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList (Duplicati.Library.Main.BackendManager backend, Duplicati.Library.Main.Options options, Duplicati.Library.Main.Database.LocalDatabase database, Duplicati.Library.Main.IBackendWriter log, System.String protectedfile) [0x001f8] in <118ad25945a24a3991f7b65e7a45ea1e>:0
at Duplicati.Library.Main.Operation.BackupHandler.PostBackupVerification () [0x00058] in <118ad25945a24a3991f7b65e7a45ea1e>:0
at Duplicati.Library.Main.Operation.BackupHandler.Run (System.String sources, Duplicati.Library.Utility.IFilter filter) [0x007f6] in <118ad25945a24a3991f7b65e7a45ea1e>:0
Jan 22, 2020 2:41 PM: Message
Found 1 files that are missing from the remote storage, please run repair
I ran repair and attempted to restart the backup, but now it always says “Fatal error: Unsupported addition of passphrase” in the GUI. No additional logfile messages are generated.
I’m not sure what this message is referring to, as I haven’t added or changed any settings, and I confirmed that the passphrase I used initially when creating the AES-encrypted backup is the same as it was before.
Is there any hope of recovering this so I can complete the backup successfully? Or should I just start over at this point? And what might the “unsupported addition of passphrase” message even mean?
“You have attempted to add a passphrase to an existing backup, which is not supported. Please configure a new clean backup if you want to add a passphrase.”
Do you have a way to look at the SFTP destination to see if .zip files exist, or do all files end with .aes? Seeing the latter would give me more hope of recovery, although missing files can also be bad news…
Did you run Repair from Database screen? It sounds like somehow the passphrase information is lost. Possibly it can be forced back on using the –allow-passphrase-change option to override current error.
One issue at a time, OR …
If you’re happy enough to start over, I’d start over on 2.0.5.1 Beta, as it’s more reliable than 2.0.4.5/23.
Choosing sizes in Duplicati may be worth reading if you start over. Large backups can grow slow at the default 100 KB –blocksize. Raising that, e.g. to 1 MB, may improve performance but deduplicate worse.
The version is quite old - 2.0.2.1_beta_2017-08-01, and I’m guessing the right thing to do is just upgrade and try again. The files at the backup destination are .zip – a little over 62,000 of them. I have tried running repair, but it doesn’t solve the problem.
That said, the reason why I haven’t is because this backup runs on an ancient CentOS 7 machine that does not like getting updated from non-core repos or with out-of-band packages. I was hoping someone would be able to say “oh, that happens when XYZ, just click this button!”, but I guess I’ll just have to rip off the band-aid. Or maybe run duplicati in a more modern docker container.
If the remote files have no .aes and are old, and web UI shows encryption on, then that’s inconsistent.
Possibly you can turn encryption off to match the actual backup. You’ll probably need to OK a warning.
If you can get Duplicati to be agreeable with your unencrypted zip files, then you could try more things.
The changelog shows some Repair improvement made in 2018, but I don’t know if that will be enough.
Missing files that are dblock files are usually actual missing source file data, but messages aren’t clear.
Restart would probably be cleanest, but I mention some other options in case you want to try them out.
Generally I advise 2.0.5.1 Beta now that it’s out, but note that it needs mono v5 or higher, typically from Mono project, per Prerequisites and Mono download directions. Or using Docker should get mono right.
If you decide to try to fix it, and if you can get the password issue solved, then possibly missing file can be somewhat solved (from an error point of view, but maybe not from a damaged backup point of view) using Repair (preferably at least 2.0.4.5’s, in case it’s better), or maybe Recreate, or maybe list-broken-files and purge-broken-files as described in Recovering by purging files. Cleanest/easiest may be startover backup.
Thanks so much for the support. I attempted a couple of things
Modifying the config to remove the AES encrypt option and password
Removing the password via the “I’m brave” option
Repairing, deleting, recreating the database
But all gave a littany of (sometimes different) errors. Since this was only the first backup I decided to just cut my losses and recreate it from scratch. I also did take the opportunity to upgrade to 2.0.5.1_beta_2020-01-18. Aside from some mono conflicts it wasn’t as bad as I thought it’d be. I’m now about 40% into a test backup of the same payload (local USB vs. over-the-network. Fingers crossed. Thanks again for all of your suggestions!
