Files missing when trying to restore

tarball · June 27, 2018, 9:43am

Hello, I am realising that the backups I run since one month seams incomplete: First the total source of my backup is 59GB while the destination is only 43GB.
On a new laptop, I tried to transfer my backup by using Restore from a config file, now, almost all my folders are present but a lot of files are missing on my new computer.
Also, I have some error, like this one when I backup, I think it is related to my problem but I have some trouble to interpret it.
“Failed while executing “Backup” with id: 2”
https://pastebin.com/raw/B9KRbXrr
Thank you in advance for your help.

System.AggregateException: One or more errors occurred.
---> System.AggregateException: Found 3 remote files that are not recorded in local storage, please run repair
---> Duplicati.Library.Interface.UserInformationException: Found 3 remote files that are not recorded in local storage, please run repair
 at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList (Duplicati.Library.Main.BackendManager backend, Duplicati.Library.Main.Options options, Duplicati.Library.Main.Database.LocalDatabase database, Duplicati.Library.Main.IBackendWriter log, System.String protectedfile) [0x00104] in <c093ff03452d49f59d25577493ea2214>:0
 at Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify (Duplicati.Library.Main.BackendManager backend, System.String protectedfile) [0x0011e] in <c093ff03452d49f59d25577493ea2214>:0
 at Duplicati.Library.Main.Operation.BackupHandler+<RunAsync>d__19.MoveNext () [0x004ed] in <c093ff03452d49f59d25577493ea2214>:0
 --- End of inner exception stack trace ---
 at Duplicati.Library.Main.Operation.BackupHandler+<RunAsync>d__19.MoveNext () [0x01018] in <c093ff03452d49f59d25577493ea2214>:0
 --- End of inner exception stack trace ---
 at CoCoL.ChannelExtensions.WaitForTaskOrThrow (System.Threading.Tasks.Task task) [0x0005d] in <6973ce2780de4b28aaa2c5ffc59993b1>:0
 at Duplicati.Library.Main.Operation.BackupHandler.Run (System.String[] sources, Duplicati.Library.Utility.IFilter filter) [0x00008] in <c093ff03452d49f59d25577493ea2214>:0
 at Duplicati.Library.Main.Controller+<>c__DisplayClass14_0.<Backup>b__0 (Duplicati.Library.Main.BackupResults result) [0x00035] in <c093ff03452d49f59d25577493ea2214>:0
 at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String[]& paths, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x0022e] in <c093ff03452d49f59d25577493ea2214>:0
 at Duplicati.Library.Main.Controller.Backup (System.String[] inputsources, Duplicati.Library.Utility.IFilter filter) [0x00068] in <c093ff03452d49f59d25577493ea2214>:0
 at Duplicati.Server.Runner.Run (Duplicati.Server.Runner+IRunnerData data, System.Boolean fromQueue) [0x00335] in <fa054d471b2b4da5877f49f4129207a7>:0
---> (Inner Exception #0) System.AggregateException: Found 3 remote files that are not recorded in local storage, please run repair
---> Duplicati.Library.Interface.UserInformationException: Found 3 remote files that are not recorded in local storage, please run repair
 at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList (Duplicati.Library.Main.BackendManager backend, Duplicati.Library.Main.Options options, Duplicati.Library.Main.Database.LocalDatabase database, Duplicati.Library.Main.IBackendWriter log, System.String protectedfile) [0x00104] in <c093ff03452d49f59d25577493ea2214>:0
 at Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify (Duplicati.Library.Main.BackendManager backend, System.String protectedfile) [0x0011e] in <c093ff03452d49f59d25577493ea2214>:0
 at Duplicati.Library.Main.Operation.BackupHandler+<RunAsync>d__19.MoveNext () [0x004ed] in <c093ff03452d49f59d25577493ea2214>:0
 --- End of inner exception stack trace ---
 at Duplicati.Library.Main.Operation.BackupHandler+<RunAsync>d__19.MoveNext () [0x01018] in <c093ff03452d49f59d25577493ea2214>:0
---> (Inner Exception #0) Duplicati.Library.Interface.UserInformationException: Found 3 remote files that are not recorded in local storage, please run repair
 at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList (Duplicati.Library.Main.BackendManager backend, Duplicati.Library.Main.Options options, Duplicati.Library.Main.Database.LocalDatabase database, Duplicati.Library.Main.IBackendWriter log, System.String protectedfile) [0x00104] in <c093ff03452d49f59d25577493ea2214>:0
 at Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify (Duplicati.Library.Main.BackendManager backend, System.String protectedfile) [0x0011e] in <c093ff03452d49f59d25577493ea2214>:0
 at Duplicati.Library.Main.Operation.BackupHandler+<RunAsync>d__19.MoveNext () [0x004ed] in <c093ff03452d49f59d25577493ea2214>:0 <---
---> (Inner Exception #1) System.AggregateException: One or more errors occurred.
---> System.Exception: Unable to find log in lookup table, this may be caused by attempting to transport call contexts between AppDomains (eg. with remoting calls)
 at Duplicati.Library.Logging.Log.get_CurrentScope () [0x0004d] in <e74474bd31bc48c3b31b195ac61abde2>:0
 at Duplicati.Library.Logging.Log.WriteMessage (Duplicati.Library.Logging.LogMessageType type, System.String tag, System.String id, System.Exception ex, System.String message, System.Object[] arguments) [0x0001e] in <e74474bd31bc48c3b31b195ac61abde2>:0
 at Duplicati.Library.Logging.Log.WriteWarningMessage (System.String tag, System.String id, System.Exception ex, System.String message, System.Object[] arguments) [0x00000] in <e74474bd31bc48c3b31b195ac61abde2>:0
 at Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess+<>c.<Run>b__1_3 (System.String rootpath, System.String errorpath, System.Exception ex) [0x00000] in <c093ff03452d49f59d25577493ea2214>:0
 at Duplicati.Library.Utility.Utility+<EnumerateFileSystemEntries>d__23.MoveNext () [0x001d9] in <5c22135b7083459abdb8767ab6361c3a>:0
 at System.Linq.Enumerable+SelectManySingleSelectorIterator`2[TSource,TResult].MoveNext () [0x0006f] in <a16ef142877c4302b96a103db87ba16b>:0
 at Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess+<ExpandWorkList>d__4.MoveNext () [0x000da] in <c093ff03452d49f59d25577493ea2214>:0
 at Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess+<>c__DisplayClass1_0+<<Run>b__0>d.MoveNext () [0x00263] in <c093ff03452d49f59d25577493ea2214>:0
 --- End of stack trace from previous location where exception was thrown ---
 at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <73ee1b14c2fa4d61b481096ff3d8d6d7>:0
 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <73ee1b14c2fa4d61b481096ff3d8d6d7>:0
 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <73ee1b14c2fa4d61b481096ff3d8d6d7>:0
 at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <73ee1b14c2fa4d61b481096ff3d8d6d7>:0
 at System.Runtime.CompilerServices.TaskAwaiter.GetResult () [0x00000] in <73ee1b14c2fa4d61b481096ff3d8d6d7>:0
 at CoCoL.AutomationExtensions+<RunTask>d__10`1[T].MoveNext () [0x000cc] in <6973ce2780de4b28aaa2c5ffc59993b1>:0
 --- End of inner exception stack trace ---
---> (Inner Exception #0) System.Exception: Unable to find log in lookup table, this may be caused by attempting to transport call contexts between AppDomains (eg. with remoting calls)
 at Duplicati.Library.Logging.Log.get_CurrentScope () [0x0004d] in <e74474bd31bc48c3b31b195ac61abde2>:0
 at Duplicati.Library.Logging.Log.WriteMessage (Duplicati.Library.Logging.LogMessageType type, System.String tag, System.String id, System.Exception ex, System.String message, System.Object[] arguments) [0x0001e] in <e74474bd31bc48c3b31b195ac61abde2>:0
 at Duplicati.Library.Logging.Log.WriteWarningMessage (System.String tag, System.String id, System.Exception ex, System.String message, System.Object[] arguments) [0x00000] in <e74474bd31bc48c3b31b195ac61abde2>:0
 at Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess+<>c.<Run>b__1_3 (System.String rootpath, System.String errorpath, System.Exception ex) [0x00000] in <c093ff03452d49f59d25577493ea2214>:0
 at Duplicati.Library.Utility.Utility+<EnumerateFileSystemEntries>d__23.MoveNext () [0x001d9] in <5c22135b7083459abdb8767ab6361c3a>:0
 at System.Linq.Enumerable+SelectManySingleSelectorIterator`2[TSource,TResult].MoveNext () [0x0006f] in <a16ef142877c4302b96a103db87ba16b>:0
 at Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess+<ExpandWorkList>d__4.MoveNext () [0x000da] in <c093ff03452d49f59d25577493ea2214>:0
 at Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess+<>c__DisplayClass1_0+<<Run>b__0>d.MoveNext () [0x00263] in <c093ff03452d49f59d25577493ea2214>:0
 --- End of stack trace from previous location where exception was thrown ---
 at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <73ee1b14c2fa4d61b481096ff3d8d6d7>:0
 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <73ee1b14c2fa4d61b481096ff3d8d6d7>:0
 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <73ee1b14c2fa4d61b481096ff3d8d6d7>:0
 at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <73ee1b14c2fa4d61b481096ff3d8d6d7>:0
 at System.Runtime.CompilerServices.TaskAwaiter.GetResult () [0x00000] in <73ee1b14c2fa4d61b481096ff3d8d6d7>:0
 at CoCoL.AutomationExtensions+<RunTask>d__10`1[T].MoveNext () [0x000cc] in <6973ce2780de4b28aaa2c5ffc59993b1>:0 
 <--- <--- <---

JonMikelV · June 28, 2018, 3:11pm

Hello @tarball, welcome to the forum!

It sounds like you’ve got a few different things going on so let’s try to address them individually.

Most likely this is because Duplicati uses deduplication so commonalities between files (and file versions) are only backed up once. On top of that, by default backups are also compressed - so depending on the type of content in your source folders you might be seeing the smaller destination size simply because of compression benefits.

If you go back to the Restore menu and search specifically for some of the files that are missing, do you find them? If so, then there may have been an issue during the restore but if not, they may not have made it into the backup in the first place (see next section)…

So that error generally means the backup failed for some reason and we’ll have to look at the logs to figure out exactly what happened so we know what to do to fix it.

If you click on the error message in the GUI, does it open up and show you additional details?

Also, what version of Duplicati are you using and on what OS?

(By the way, I edited your topic by adding your pastebin text directly to your post so it can be more easily found with the forum search.)

tarball · June 28, 2018, 3:40pm

Thank you for your answer.

Ok, that makes sense.

I am trying to do so but now I get: “The database file is locked”, I hope it is not an extra problem!

I always have warnings but they seem to be old. The most recent logs are generally in About > History. That is where I found the log file I attached in the previous post.

The warning I get usually looks like that:

DeletedFiles: 0 
DeletedFolders: 0 
ModifiedFiles: 0 
ExaminedFiles: 444683 
OpenedFiles: 0 
AddedFiles: 0 
SizeOfModifiedFiles: 0 
SizeOfAddedFiles: 0 
SizeOfExaminedFiles: 63672286863 
SizeOfOpenedFiles: 0 
NotProcessedFiles: 0 
AddedFolders: 0 
TooLargeFiles: 0 
FilesWithError: 0 
ModifiedFolders: 0 
ModifiedSymlinks: 0 
AddedSymlinks: 0 
DeletedSymlinks: 0 
PartialBackup: False 
Dryrun: False 
MainOperation: Backup 
CompactResults: [
	DeletedFileCount: 2 
	DownloadedFileCount: 0 
	UploadedFileCount: 0 
	DeletedFileSize: 194010 
	DownloadedFileSize: 0 
	UploadedFileSize: 0 
	Dryrun: False 
]
MainOperation: Compact 
ParsedResult: Warning 
Version: 2.0.3.7 (2.0.3.7_canary_2018-06-17) 
EndTime: 28/06/2018 17:09:21 (1530198561) 
BeginTime: 28/06/2018 17:08:45 (1530198525) 
Duration: 00:00:35.4603900 
Messages: [ 
	2018-06-28 16:52:48 +02 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: L'opération Backup a débuté, 
	2018-06-28 16:53:13 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: (), 
	2018-06-28 16:53:26 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (1,72 KB), 
	2018-06-28 17:07:40 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b18ee603ecfae4a028eec4fb662e9a59c.dblock.zip.aes (168,97 KB), 
	2018-06-28 17:08:18 +02 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-b18ee603ecfae4a028eec4fb662e9a59c.dblock.zip.aes (168,97 KB), 
	... 
] 
Warnings: [ 
	2018-06-28 16:53:26 +02 - [Warning-Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess-FileAccessError]: Error reported while accessing file: /censured, 
	2018-06-28 16:53:26 +02 - [Warning-Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess-FileAccessError]: Error reported while accessing file: /censured, 
	2018-06-28 16:53:26 +02 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: /censured (ETC)/UTC--2016-02-10T12-31-45.967997828Z--3a400429118dcac9c3f3c9e44543508bcf494c35, 
	2018-06-28 16:53:26 +02 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: /censured/UTC--2016-07-27T17-10-14.741900874Z--f6b580a58158a69ca2f65ea0132ccb3c04d71209, 
	2018-06-28 16:53:26 +02 - [Warning-Duplicati.Library.Main.Operation.Backup.FileBlockProcessor.FileEntry-PathProcessingFailed]: Failed to process path: /censured (ETC)/UTC--2016-03-15T13-52-19.889781978Z--d78dfb90d445d996839670a8fcba499c945fd911,
	...
]

JonMikelV · June 28, 2018, 4:00pm

Probably not - that usually happens when Duplicati is running a backup, can check if it’s doing so?

These warnings usually happen due to permissions issues on your machine - for example, if you run Dulicati under your user account and you are not an administrator Duplicati would likely not have access rights to certain system folders or other user files.

These errors happen when there are files in the destination location that Duplicati doesn’t expect. This cane be caused by situations including:

something / somebody else put the files there (so they are not really Duplicati backup files) in which case you’ll need to move those files (anything NOT starting with “duplicati-”) out of the destination folder
the local Duplicati database is out of sync with what’s at the destination in which case running a database Repair should resolve it issue

In the GUI logs there should be additional details listing the unexpected files. If you can get those and post them here that might help narrow down the source of the problem.

tarball · June 29, 2018, 9:06am

Yes it does, to be exact It switches between “running backup” to “Verifying backend data”. This last state is unexpectedly long (about more than 10min)

Ok. So if they are just warnings, I can just ignore it for now? I will just not be able to find back those files but they are not stopping the process, right?

I think I will start the process from zero in order to have a clean backup. I think your answers helped me to understand the issue: it may be due to my new computer: I started Duplicati on it as well and I think it backed up empty folders, leading to an out of sync status.

tarball · June 30, 2018, 9:52am

After some extra trials, I have a weird situation where my old computer has an available backup dating from 2 days ago and my new computer with the same config can only access an older version dating from 3 days ago. I am using a Duplicati - 2.0.3.3_beta_2018-04-02 on linux (Debian). My remote server is a Backbaze B2 Cloud Storage.
What can cause this differences of available backups between the two machines? How can I solve that?

JonMikelV · July 2, 2018, 3:12am

Just to make sure I’m understanding your situation correctly…

Assuming your old computer was backing up to B2 in bucket / folder (let’s call it) A, when you set up your new computer you also told it to use B2 bucket / folder A presumably so you could restore your files and keep the version history.

Are you now running backups on BOTH computers to the same destination folder?

tarball · July 2, 2018, 7:45am

Yes that is correct. Both computers are based on the same config file and has the exact same destination folder. That is why I don’t understand why the backup lists are not the same on both computers.

Comupturer 1 (old)

Comupturer 2 (new)

[edit] I am launching a brand new backup on computer 1 to see if it’s work.

tarball · July 2, 2018, 8:58am

The fresh backup I tried to make shows this error: It says that the disk is full. Does it mean the server side disk is full?

with remoting calls) at Duplicati.Library.Logging.Log.get_CurrentScope () [0x0004d] in <e74474bd31bc48c3b31b195ac61abde2>:0 at Duplicati.Library.Logging.Log.WriteMessage (Duplicati.Library.Logging.LogMessageType type, System.String tag, System.String id, System.Exception ex, System.String message, System.Object[] arguments) [0x0001e] in <e74474bd31bc48c3b31b195ac61abde2>:0 at Duplicati.Library.Logging.Log.WriteWarningMessage (System.String tag, System.String id, System.Exception ex, System.String message, System.Object[] arguments) [0x00000] in <e74474bd31bc48c3b31b195ac61abde2>:0 at Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess+<>c.<Run>b__1_3 (System.String rootpath, System.String errorpath, System.Exception ex) [0x00000] in <c093ff03452d49f59d25577493ea2214>:0 at Duplicati.Library.Utility.Utility+<EnumerateFileSystemEntries>d__23.MoveNext () [0x001d9] in <5c22135b7083459abdb8767ab6361c3a>:0 at System.Linq.Enumerable+SelectManySingleSelectorIterator`2[TSource,TResult].MoveNext () [0x0006f] in <a16ef142877c4302b96a103db87ba16b>:0 at Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess+<ExpandWorkList>d__4.MoveNext () [0x000da] in <c093ff03452d49f59d25577493ea2214>:0 at Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess+<>c__DisplayClass1_0+<<Run>b__0>d.MoveNext () [0x00263] in <c093ff03452d49f59d25577493ea2214>:0 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <73ee1b14c2fa4d61b481096ff3d8d6d7>:0 at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <73ee1b14c2fa4d61b481096ff3d8d6d7>:0 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <73ee1b14c2fa4d61b481096ff3d8d6d7>:0 at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <73ee1b14c2fa4d61b481096ff3d8d6d7>:0 at System.Runtime.CompilerServices.TaskAwaiter.GetResult () [0x00000] in <73ee1b14c2fa4d61b481096ff3d8d6d7>:0 at CoCoL.AutomationExtensions+<RunTask>d__10`1[T].MoveNext () [0x000cc] in <6973ce2780de4b28aaa2c5ffc59993b1>:0 <--- <--- <---

[Edit] It appears my local disk is full. The temp files created by Duplicati seams to be big enough to fill my disk. After deleting some data on my hard drive, the message is no longer shown.

[Edit 2] I keep having this disk is full alert that stops the backup. I do my best to free disk space but it seems to be never enough. It is weird because my disk has 114GB free.

Disk full. Path /tmp/dup-96b99f50-a326-4a9f-962e-b4f7d0149b9e

[Edit 3] I fixed the “disk is full” alert by entering a new temp dir option, pointing to something else than /tmp/ and setting the right rights to that folder in order to let Duplicati write in it. /tmp/ was considered like a drive or a media (I admit not understanding this part) and was full indeed.
I am now launching a new backup.

JonMikelV · July 5, 2018, 3:10pm

I’m not even sure how your backups are running if they’re both using the same destination folder. One should be working fine but the other should be showing an error similar to “unexpected files”.

There was an issue with not cleaning up temp files that started in version 2.0.3.6 but it was fixed in 2.0.3.8. If you’re still using 2.0.3.3 then I’m not sure why you’d be running into this issue unless you have a large --blocksize (Upload volume size), --asynchronous-upload-limit, or very low disk space. (Note that by default Duplicati will need a bit more than 4 times your Upload volume size in temp storage.)

Thanks for the screen shot - that clarifies that you are seeing missing VERSIONS to be restored from (I thought you meant when you did a restore some of the files were missing from it.)

Remember that Duplicati keeps a local record of what’s been backed up, so if you ran a Computer 2 backup on June 27 and a Computer 1 backup on June 28 then the LOCAL record of each computer would only know of the backup run on that computer.

If you use the main menu Restore link and choose “Direct restore from backup files…” it should use ONLY the files in your destination to generate the list of versions that are available to be restored.

Try that and see if things work more like you expect.