Yes, you have to use the --allow-full-removal option for it to proceed.
As an alternative, it’s probably faster to manually delete the files on the S3 side using the AWS console and then deleting the local job database (click the backup job, click the blue Database link, then click the blue Delete button). Once you do both steps you can run the backup again and it will start over from the beginning.
I performed a [Recreate (delete and repair)]
Hi drwtsn32,
It has been running the Database recreate for 7 days or so and I am wondering if it is stuck or just really slow.
Here are the verbose output logs if this helps:
Recreating database …
Jul 16, 2020 8:39 AM: Backend event: Get - Completed: duplicati-b03f23ff125d644efaf8471a142b86c8b.dblock.zip.aes (49.98 MB)
Jul 16, 2020 8:38 AM: Pass 3 of 3, processing blocklist volume 316 of 23044
Jul 16, 2020 8:38 AM: Backend event: Get - Started: duplicati-b03f23ff125d644efaf8471a142b86c8b.dblock.zip.aes (49.98 MB)
Jul 16, 2020 8:22 AM: Backend event: Get - Completed: duplicati-b03f1d1134be943cf98dfba788b6bd692.dblock.zip.aes (49.93 MB)
Jul 16, 2020 8:21 AM: Pass 3 of 3, processing blocklist volume 315 of 23044
I thought the plan was to start over… if so I would not bother with database recreation. Stop that process, delete all the files related to this job in the S3 bucket (sounds like you didn’t do that step), and then just Delete the database (do not use recreate). When you next run a backup job, it will behave as if it’s the first backup and will create a new, blank database.
Sorry for the confusion, I was able to verify the database is cleaned up.
I am now assuming I just wait until the next scheduled backup.
I will post an update after 1230pm tomorrow
thanks again
You can wait or you can kick off the backup manually. Good luck!
I actually tried, but it threw 2 new errors.
Error while running Vembu2AWS
Found 46657 remote files that are not recorded in local storage, please run repair
Jul 17, 2020 8:10 AM: The operation Backup has failed with error: Found 46657 remote files that are not recorded in local storage, please run repair
{“ClassName”:“Duplicati.Library.Interface.UserInformationException”,“Message”:“Found 46657 remote files that are not recorded in local storage, please run repair”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:" at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList(BackendManager backend, Options options, LocalDatabase database, IBackendWriter log, String protectedfile)\r\n at Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify(BackendManager backend, String protectedfile)\r\n at Duplicati.Library.Main.Operation.BackupHandler.d__20.MoveNext()\r\n— End of stack trace from previous location where exception was thrown —\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)\r\n at Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.b__0(BackupResults result)\r\n at Duplicati.Library.Main.Controller.RunAction[T](T result, String& paths, IFilter& filter, Action`1 method)\r\n at Duplicati.Library.Main.Controller.Backup(String inputsources, IFilter filter)\r\n at Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)",“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:“8\nVerifyRemoteList\nDuplicati.Library.Main, Version=2.0.5.1, Culture=neutral, PublicKeyToken=null\nDuplicati.Library.Main.Operation.FilelistProcessor\nVoid VerifyRemoteList(Duplicati.Library.Main.BackendManager, Duplicati.Library.Main.Options, Duplicati.Library.Main.Database.LocalDatabase, Duplicati.Library.Main.IBackendWriter, System.String)”,“HResult”:-2146233088,“Source”:“Duplicati.Library.Main”,“WatsonBuckets”:null}
Jul 17, 2020 8:10 AM: Fatal error
{“ClassName”:“Duplicati.Library.Interface.UserInformationException”,“Message”:“Found 46657 remote files that are not recorded in local storage, please run repair”,“Data”:null,“InnerException”:null,“HelpURL”:null,“StackTraceString”:" at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList(BackendManager backend, Options options, LocalDatabase database, IBackendWriter log, String protectedfile)\r\n at Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify(BackendManager backend, String protectedfile)\r\n at Duplicati.Library.Main.Operation.BackupHandler.d__20.MoveNext()\r\n— End of stack trace from previous location where exception was thrown —\r\n at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n at CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)\r\n at Duplicati.Library.Main.Controller.<>c__DisplayClass14_0.b__0(BackupResults result)\r\n at Duplicati.Library.Main.Controller.RunAction[T](T result, String& paths, IFilter& filter, Action`1 method)",“RemoteStackTraceString”:null,“RemoteStackIndex”:0,“ExceptionMethod”:“8\nVerifyRemoteList\nDuplicati.Library.Main, Version=2.0.5.1, Culture=neutral, PublicKeyToken=null\nDuplicati.Library.Main.Operation.FilelistProcessor\nVoid VerifyRemoteList(Duplicati.Library.Main.BackendManager, Duplicati.Library.Main.Options, Duplicati.Library.Main.Database.LocalDatabase, Duplicati.Library.Main.IBackendWriter, System.String)”,“HResult”:-2146233088,“Source”:“Duplicati.Library.Main”,“WatsonBuckets”:null}
Looks like you still haven’t deleted the files for this backup on the back end (S3 bucket). Did you try? Be careful of course if this is a shared S3 bucket used by other backups or for other purposes - only delete the files related to this particular backup.
I removed the folder and recreated it on S3, but still continue to receive the same error.
That shouldn’t be… Duplicati shouldn’t be detecting files if you really removed them. Can you double check the destination as entered into Duplicati and compare with what the AWS console shows for that S3 bucket?
My boss jumped in and fixed it. Not sure what he did yet.
I’ll let you know tomorrow after 1230, in the event it fails again.