OK. Recreate database finished in 9.5 hours. The new database is 5.1 GB.
After that I changed Backup Retention from “Keep all backups” to ”Smart Backup Retention” and then I did a “Compact now”. I expected the backup to shrink considerably, but nothing happened. Is there some way I can delete all the unwanted versions of all the files in the backup?
Compact alone doesn’t prune versions (as far as I know). Compact simply repackages volumes to use space more efficiently. Backup version pruning happens after a regular backup job. So try running a backup job and you will probably see it go to the “deleting versions” phase at the end of the backup job.
After the backup job, Duplicati may decide to do an auto compaction. If it does not, you can certainly run compact again.
You should see a reduction in the back end storage size afterwards. The amount of space recovered depends on a lot of factors.
1 Like
Thanks again for your help!
Now it’s been running for about 40 hours, saying “Deleting unwanted files …”, with the ‘mono-sgen’ process using about 25% of my CPUs. It would be nice to have some idea about how far it is in the process, but I can’t see any more info.
If the developer(s) read this: Please add some counter that shows how far through the process it is, because I can’t really now if it is stuck in an infinite loop somewhere.
But there are 566 backups, totaling 760 GB.
I can add that my PC sends and receives very little. I guess around 180 KB each 20 seconds, which is a bit worrying. 
Go to About → Show Log → Live → Verbose … you should see log entries update regularly there to confirm that it’s not stuck.
Most of the work is probably with SQL queries to determine which blocks are referenced by any backup jobs. So it’s not really a bandwidth issue, more of a CPU issue. And I believe SQL queries are limited to using only one CPU core, unfortunately.
What kind of processor is in this machine?
Something is wrong here.
At first “Disabled” is selected by default in About -> Show Log -> Live.
Then I choose something else, but no matter what I choose, I get entries that are from before the current backup job… And it even stays on the screen although I go back to “Disabled”…
Should I logout and login?
I have 4 CPU cores, and mono-sgen never uses more than 27%.
Leave it on Verbose for a while and wait for the next log entry to appear. It may be a while between log entries. And yes, it is normal for Duplicati to only be able to use one core (~25% cpu usage on a 4 core machine) for certain tasks.
Finally the process has ended. The backup has not gotten smaller. I guess I need to do the “Compact now”.
But first: There is this error message:
“Error while running Lenovo-Ubuntu
Value cannot be null. Parameter name: path”
There doesn’t seem to be any more details.
Can anyone tell me what that means?
Is there any more detail in About -> Show Log -> Stored? (May need to click the entries to expand.)
Yes! There are 3 new entries:
Feb 1, 2020 2:39 AM: Failed while executing “Backup” with id: 1:
Duplicati.Library.Interface.UserInformationException: Found 1 remote files that are not recorded in local storage, please run repair
(I have deleted the rest.)
Feb 1, 2020 2:37 AM: Failed while executing “Backup” with id: 1:
Duplicati.Library.Interface.UserInformationException: Found 1 remote files that are not recorded in local storage, please run repair
(I have deleted the rest.)
I am sorry this log entry is very long! What does it mean:
Feb 1, 2020 2:35 AM: Failed while executing “Backup” with id: 1
System.ArgumentNullException: Value cannot be null.
Parameter name: path
at Duplicati.Library.Main.BackendManager.WaitForEmpty (Duplicati.Library.Main.Database.LocalDatabase db, System.Data.IDbTransaction transation) [0x00067] in <8f1de655bd1240739a78684d845cecc8>:0
at Duplicati.Library.Main.Operation.CompactHandler.DoDelete (Duplicati.Library.Main.Database.LocalDeleteDatabase db, Duplicati.Library.Main.BackendManager backend, System.Collections.Generic.IEnumerable1[T] deleteableVolumes, System.Data.IDbTransaction& transaction) [0x00040] in <8f1de655bd1240739a78684d845cecc8>:0 at Duplicati.Library.Main.Operation.CompactHandler.DoCompact (Duplicati.Library.Main.Database.LocalDeleteDatabase db, System.Boolean hasVerifiedBackend, System.Data.IDbTransaction& transaction, Duplicati.Library.Main.BackendManager sharedBackend) [0x004a2] in <8f1de655bd1240739a78684d845cecc8>:0 at Duplicati.Library.Main.Operation.DeleteHandler.DoRun (Duplicati.Library.Main.Database.LocalDeleteDatabase db, System.Data.IDbTransaction& transaction, System.Boolean hasVerifiedBacked, System.Boolean forceCompact, Duplicati.Library.Main.BackendManager sharedManager) [0x00397] in <8f1de655bd1240739a78684d845cecc8>:0 at Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired (Duplicati.Library.Main.BackendManager backend, System.Int64 lastVolumeSize) [0x000a5] in <8f1de655bd1240739a78684d845cecc8>:0 at Duplicati.Library.Main.Operation.BackupHandler.RunAsync (System.String[] sources, Duplicati.Library.Utility.IFilter filter, System.Threading.CancellationToken token) [0x01033] in <8f1de655bd1240739a78684d845cecc8>:0 at CoCoL.ChannelExtensions.WaitForTaskOrThrow (System.Threading.Tasks.Task task) [0x00050] in <9a758ff4db6c48d6b3d4d0e5c2adf6d1>:0 at Duplicati.Library.Main.Operation.BackupHandler.Run (System.String[] sources, Duplicati.Library.Utility.IFilter filter, System.Threading.CancellationToken token) [0x00009] in <8f1de655bd1240739a78684d845cecc8>:0 at Duplicati.Library.Main.Controller+<>c__DisplayClass14_0.<Backup>b__0 (Duplicati.Library.Main.BackupResults result) [0x0004b] in <8f1de655bd1240739a78684d845cecc8>:0 at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String[]& paths, Duplicati.Library.Utility.IFilter& filter, System.Action1[T] method) [0x0026f] in <8f1de655bd1240739a78684d845cecc8>:0
at Duplicati.Library.Main.Controller.Backup (System.String inputsources, Duplicati.Library.Utility.IFilter filter) [0x00074] in <8f1de655bd1240739a78684d845cecc8>:0
at Duplicati.Server.Runner.Run (Duplicati.Server.Runner+IRunnerData data, System.Boolean fromQueue) [0x00349] in :0
Now I have run “Compact now” several times (and a few “Repair database”), but Duplicati still says Backup: 761 GB.
Why?
Is the compact completing successfully? No more errors?
Oops! Now suddenly Duplicati says:
Backup: 271.92 GB
Apparently after yet another ordinary backup.
So finally everything is as expected, although the process has been quite cumbersome.
There were no errors, only warnings.
Interesting that the stats weren’t updated until the next backup. Not sure if that is intentional or not. Glad you have finally achieved your goal!
Yes! Thank you for everything!
And for the first time in about a week the ordinary scheduled backup succeeded!
1 Like