I have a large (1.5TB) backup set Jottacloud. When I try to do a full remote verify, I get the following error:
ErrorID: MissingRemoteFiles
Found 2 files that are missing from the remote storage, please run repair
Return code: 100
I have tried a purge command, with --purge-broken-files. I have tried a repair command, with and without --purge-broken-files. None of these fix the issue.
How can I resolve this, ideally without deleting my old backup versions?
Hello
I think that logs should provide you with the missing files names. Then you could search the affected files with the appropriate Duplicati function. Then consider what to do, depending on if there is important data that is only concerned by these files, or not. Sometimes removing the references to backup files without any practical interest is the best approach.
Hi gpatel-fr. Thanks, I can see which files are affected, but I don’t know what to do about them. How can I remove the references to those files?
You can always delete files on the backend,
but it’s necessary first to assess what will be lost. If the concerned files (on the backend) still exist on the source system in the same version it’s not a great loss, Duplicati will backup them again next time. If not that’s more annoying. You should then see if the missing files exist in other versions (more recent) and if they matter at all.
My understanding of the DELETE command is that it deletes files from the backup sets, not missing files from the target storage. The missing files are:
Missing file: duplicati-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.dblock.zip.aes
Missing file: duplicati-XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX.dblock.zip.aes
Recovering by purging files run in GUI Commandline removes backup of files affected by dblock loss.
The AFFECTED command can be given either or both dblock names for a second opinion on impact.
Anything preceding the loss that might hint at cause? You don’t want destination losing files too often.
Thanks. I tried the purge-broken-files command, but it did nothing, and a subsequent verify command failed with the same error message as before.
I tried another repair, got:
Failed to perform cleanup for missing file: duplicati-b9fbade3c7e8a4ca68e0c13232d8a4a6c.dblock.zip.aes, message: Repair not possible, missing 50 blocks.
If you want to continue working with the database, you can use the "list-broken-files" and "purge-broken-files" commands to purge the missing data from the database and the remote storage. => Repair not possible, missing 50 blocks.
If you want to continue working with the database, you can use the "list-broken-files" and "purge-broken-files" commands to purge the missing data from the database and the remote storage.
Trying list-broken-files gives
Listing remote folder ...
Return code: 0
Same with purge-broken-files.
What about the affected command described earlier? Maybe those files don’t aren’t actually in use now.
Any comment at all about history leading up to error? Your logs have something, or you may remember.
Which command? Do you mean affected?
The following related log messages were found:
21/05/2023 19:04:44: {"Size":521014877,"Hash":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXX="}
Return code: 0
The only history leading up to this was that the machine runs an automatic backup at night. I got an error message related to rclone and Jottacloud authentication, which I fixed and re-ran the backup, resulting in this error message.
It looks like perhaps the authentication token expired during the backup, resulting in some files not being uploaded.
It also tried to delete the last backup, version 0, but got the same error message about missing files.
Yes, but assuming you posted the output (correct?) it’s not finding anything. In a sense, that’s good.
Are you running the rclone workaround for Jottacloud’s change? If not, how would rclone fit into this?
Although the tests you’ve done strongly suggest the files got recorded but don’t have anything useful, proving that is kind of tough unless someone goes looking in the database, e.g. you with DB browser, following some directions I can post, or you first do Creating a bug report and post a link to an upload.
I can look in there and see if I can see what’s wrong, maybe how it went wrong, and what to do for fix.
If neither of those sound good, it’s also possible to kind of act based on guess. Do you routinely test if database recreate is possible? That might lead to a plan like delete those two and see if it still does… Rename your old database (rather than delete) just in case the recreate can’t be made to actually run.
Yes, I am using rclone until Duplicati’s native support for Jottacloud is fixed. It’s been working fine for quite some time, and with other backup jobs.
Thanks for offering to take a look. I will give recreating the database a try first. I should probably file a bug report too.
Doing a repair seems to have done the trick and fixed it.
I started a full remote verify, but it seem have to hung about half way through. Duplicati seems to be quite bad at dealing with temporary network issues, like VPN reconnection. I’ll try again. The error usually comes up before things really start processing though so it does seem to have gone.
Use the number-of-retries, retry-delay and new retry-with-exponential-backoff options
which made its Beta appearance in 2.0.7.1 along with a Jottacloud fix that you might want to try out, including the above options. When going through rclone, it probably has its own settings and issues.
Thanks. I just updated to 2.0.7.1, and tried native Jottacloud support. When I click on “test” is sit there forever.
I’m trying another verify with those options you suggested using rclone.
Edit: I got Jottacloud native support to work, and it seems a lot faster than using rclone. Hopefully it will complete this time, although I already got one error about a file not being retrievable.
Too vague. If you keep getting these, watch on About → Show log → Live → Retry for details.
I will get you details once the operation completes. For now it has scrolled off the page. This time the verification has not got stuck, but there is a lot of data so it’s still going.
Ugh, looks like the Jottacloud support is still broken. Eventually I started getting authentication errors. I’ll have to go back to rclone, but that seems less stable.
Update. The errors are gone when running a backup now, so that’s good.
The issue is that I can’t test the backup because it always fails part way through with both Jottacloud native and rclone.
I’ll try downloading the files manually at some point, and then using a local target to do the test. I need a spare 1.7TB to do it though.
Too vague. Got any more on that?
It just hangs downloading one of the larger files. Can’t be aborted, I have to manually kill the Duplicati task. No error logs. I assume it’s rclone giving some kind of error or hanging.