New inexplicable crash of my backup and no way to solve the problem!


#1

After a new, trouble-free period, my backups have ended again in error :rage:

I started by repairing the database, which also failed. I then tried to recreate the database, and again it failed with the following error message:

2019-02-07_08-40-17

So I followed the @JonMikelV indication in his post How to list / purge broken files.

With the “–full-result” option, I get an important list of files all located in the same subdirectory, but with “–console-log-level = verbose”, I get a completely different result, without any file from the results list of the “–full-result” option!? Where is the error? And what to do now?

Should I run the purge-broken-files command with the “–full-result” option?

I agree with @rinaldi6109 who says that

Duplicati is very very very far from being user friendly in case of errors, which in my experience happen often working with cloud destinations.

It is really high time that the developers of Duplicati worry about it! :frowning_face:


#2

Finally, I decided to run the purge-broken-files command with the “–full-result” option.

Hereafter the result :

System.Exception: Unexpected number of remote volumes marked as deleted. Found 1 filesets, but 5 volumes
   à Duplicati.Library.Main.Database.LocalDeleteDatabase.<DropFilesetsFromTable>d__5.MoveNext()
   à System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   à System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   à Duplicati.Library.Main.Operation.PurgeFilesHandler.DoRun(LocalPurgeDatabase db, IFilter filter, Action`3 filtercommand, Single pgoffset, Single pgspan)
   à Duplicati.Library.Main.Operation.PurgeBrokenFilesHandler.Run(IFilter filter)
   à Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)
   à Duplicati.Library.Main.Controller.PurgeBrokenFiles(IFilter filter)
   à Duplicati.CommandLine.Commands.PurgeBrokenFiles(TextWriter outwriter, Action`1 setup, List`1 args, Dictionary`2 options, IFilter filter)
   à Duplicati.CommandLine.Program.ParseCommandLine(TextWriter outwriter, Action`1 setup, Boolean& verboseErrors, String[] args)
   à Duplicati.CommandLine.Program.RunCommandLine(TextWriter outwriter, TextWriter errwriter, Action`1 setup, String[] args)
Return code: 100

What does it mean??

What to do next?

I am desperate for this situation where the more I advance, the less I find help or solution … :disappointed:


#3

Are you using a custom retention policy?

If so, does the error go away if you set it to “Keep all backups”? (Make note of your current custom policy so you can put after this test.)


#4

@JonMikelV, What do you mean by “custom retention policy” ?

Actually, the backup concerns my User folder on my computer. I filter some subfolders and exclude hidden files, system files and temporary files:

2019-02-09_16-48-51

Are these filters and exclusions what you consider as “custom retention policy”?

Thank you for your help.


#5

Retention policy can be set on the “Options” screen of the GUI, Step 5.

There are two places.

First under “General Options” there is a setting “Backup retention” where the default is “keep all backups”

That GUI convenience option here is converted into the second form, below (--retention-policy)

The second place to set retention policy is on that same screen (Step 5, Options) under “Advanced Options” in that long list there’s a “retention-policy” (--retention-policy on the command line). Default value is empty/unused which means keep all backups. Looks like this:

Documented here (same text as above image snap).

So, yeah, if you didn’t set this intentionally (or by accident?) then it should have defaulted to “keep all backups”


#6

@johnvk Thank you for this information; actually, the backup retention was set to “smart backup retention”.

I changed it to “Keep all backups” as suggested by @JonMikelV but with or without the “–full-result” option, the result of the purge-broken files command remains the same!?

What to do next?


#7

I believe this is caused by a mismatch between b actual dindex files and what the database thinks is there.

Are the number of restorable versions different for both an in-job and a direct-from-destination restore?


#8

10 or 5 versions (??) with in-job restore:
screenshot1
screenshot2

9 version with “Direct restore from backup files…”:

screenshot3

Is there a way to recover all this mess?


#9

Can you locate and provide any log or other information from (or maybe leading up to) the original error? Agreed recovery had more mess, but knowing what recovery is having to fix might help guide the efforts.

Errors could be in email to you, or Job --> Show log --> General, or About --> Show log --> General. Any line with about the right time can be clicked to see if it will open to reveal any details of what went wrong.

Version count confusion might explain error “Unexpected number of remote volumes marked as deleted” which says its records had extra versions at the destination. This shows up in the restore dropdown too, where smart retention allowed more than 1 version per week. Possibly something went wrong in delete?

You could look directly on the destination to see how many dated files you have with dlist in their name.
Ideally we’d have a –log-file with –log-file-log-level=Information or higher. Job --> Show log -->Remote is close, and could be helpful if we have to sort out further discrepancies between records and destination.

Things could possibly be made consistent by using the delete command on the excess versions, but I’d prefer not trying that without saving off copies of the database (job Database menu) and destination files. Recovery (especially from an unknown initial failure – thus I ask) does not always work first time around.

Sometimes people who are in a hurry to get backups running just export the job, import, and backup to a different destination. This gives them the option of either trying to fix the old one (under less pressure), or simply keeping it around as-is (no repair attempt) in case a restore of an old version is needed someday.

The latter is what I did once, but I rely on Duplicati mainly for short-term recovery not long-term retrievals.

What would your recovery priorities be? Of course, it would have been even better if things hadn’t died… That’s another reason I’m looking for what went wrong. Ideally this should be fixed, not recovered from…


#10

The last successful backup was made on 31/01/2019 and the first time it failed was on 01/02/2019. Here are the error messages of the different backup attempts since the first failure until I attempted a database repair:

Feb 1, 2019 12:47 PM: Failed while executing "Backup" with id: 3
System.NotSupportedException: Le flux ne prend pas en charge les opérations de lecture ou d'écriture d'E/S simultanées.

   à Duplicati.Library.Main.BackendManager.WaitForEmpty(LocalDatabase db, IDbTransaction transation)

   à Duplicati.Library.Main.Operation.CompactHandler.DoDelete(LocalDeleteDatabase db, BackendManager backend, IEnumerable`1 deleteableVolumes, IDbTransaction& transaction)

   à Duplicati.Library.Main.Operation.CompactHandler.DoCompact(LocalDeleteDatabase db, Boolean hasVerifiedBackend, IDbTransaction& transaction, BackendManager sharedBackend)

   à Duplicati.Library.Main.Operation.DeleteHandler.DoRun(LocalDeleteDatabase db, IDbTransaction& transaction, Boolean hasVerifiedBacked, Boolean forceCompact, BackendManager sharedManager)

   à Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired(BackendManager backend, Int64 lastVolumeSize)

   à Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__19.MoveNext()

--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée ---

   à System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   à CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)

   à Duplicati.Library.Main.Controller.<>c__DisplayClass13_0.<Backup>b__0(BackupResults result)

   
Feb 1, 2019 5:17 PM: Failed while executing "Backup" with id: 3
System.NotSupportedException: Le flux ne prend pas en charge les opérations de lecture ou d'écriture d'E/S simultanées.

   à Duplicati.Library.Main.BackendManager.WaitForEmpty(LocalDatabase db, IDbTransaction transation)

   à Duplicati.Library.Main.Operation.CompactHandler.DoDelete(LocalDeleteDatabase db, BackendManager backend, IEnumerable`1 deleteableVolumes, IDbTransaction& transaction)

   à Duplicati.Library.Main.Operation.CompactHandler.DoCompact(LocalDeleteDatabase db, Boolean hasVerifiedBackend, IDbTransaction& transaction, BackendManager sharedBackend)

   à Duplicati.Library.Main.Operation.DeleteHandler.DoRun(LocalDeleteDatabase db, IDbTransaction& transaction, Boolean hasVerifiedBacked, Boolean forceCompact, BackendManager sharedManager)

   à Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired(BackendManager backend, Int64 lastVolumeSize)

   à Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__19.MoveNext()

--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée ---

   à System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   à CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)

   à Duplicati.Library.Main.Controller.<>c__DisplayClass13_0.<Backup>b__0(BackupResults result)

   à Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)

   à Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)

   à Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)

   à Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)

   à Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)

   à Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)

   
Feb 4, 2019 12:31 PM: Failed while executing "Backup" with id: 3
Duplicati.Library.Interface.UserInformationException: Found 6 files that are missing from the remote storage, please run repair

   à Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList(BackendManager backend, Options options, LocalDatabase database, IBackendWriter log, String protectedfile)

   à Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify(BackendManager backend, String protectedfile)

   à Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__19.MoveNext()

--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée ---

   à System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   à CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)

   à Duplicati.Library.Main.Controller.<>c__DisplayClass13_0.<Backup>b__0(BackupResults result)

   à Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)

   à Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)

   à Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)

   
Feb 4, 2019 1:31 PM: Failed while executing "Backup" with id: 3
System.Net.WebException: Le serveur distant a retourné une erreur : (503) Serveur non disponible.

   à Duplicati.Library.Main.BackendManager.WaitForEmpty(LocalDatabase db, IDbTransaction transation)

   à Duplicati.Library.Main.Operation.DeleteHandler.DoRun(LocalDeleteDatabase db, IDbTransaction& transaction, Boolean hasVerifiedBacked, Boolean forceCompact, BackendManager sharedManager)

   à Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired(BackendManager backend, Int64 lastVolumeSize)

   à Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__19.MoveNext()

--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée ---

   à System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   à CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)

   à Duplicati.Library.Main.Controller.<>c__DisplayClass13_0.<Backup>b__0(BackupResults result)

   à Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)

   à Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)

   à Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)

   
Feb 5, 2019 1:48 PM: Failed while executing "Backup" with id: 3
Duplicati.Library.Interface.FileMissingException: Le fichier demandé n'existe pas

   à Duplicati.Library.Main.AsyncDownloader.AsyncDownloaderEnumerator.AsyncDownloadedFile.get_TempFile()

   à Duplicati.Library.Main.Operation.CompactHandler.DoCompact(LocalDeleteDatabase db, Boolean hasVerifiedBackend, IDbTransaction& transaction, BackendManager sharedBackend)

   à Duplicati.Library.Main.Operation.DeleteHandler.DoRun(LocalDeleteDatabase db, IDbTransaction& transaction, Boolean hasVerifiedBacked, Boolean forceCompact, BackendManager sharedManager)

   à Duplicati.Library.Main.Operation.BackupHandler.CompactIfRequired(BackendManager backend, Int64 lastVolumeSize)

   à Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__19.MoveNext()

--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée ---

   à System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   à CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)

   à Duplicati.Library.Main.Controller.<>c__DisplayClass13_0.<Backup>b__0(BackupResults result)

   à Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)

   à Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)

   à Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)
   
   
Feb 6, 2019 12:31 PM: Failed while executing "Backup" with id: 3
Duplicati.Library.Interface.UserInformationException: Found 5 files that are missing from the remote storage, please run repair

   à Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList(BackendManager backend, Options options, LocalDatabase database, IBackendWriter log, String protectedfile)

   à Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify(BackendManager backend, String protectedfile)

   à Duplicati.Library.Main.Operation.BackupHandler.<RunAsync>d__19.MoveNext()

--- Fin de la trace de la pile à partir de l'emplacement précédent au niveau duquel l'exception a été levée ---

   à System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   à CoCoL.ChannelExtensions.WaitForTaskOrThrow(Task task)

   à Duplicati.Library.Main.Controller.<>c__DisplayClass13_0.<Backup>b__0(BackupResults result)

   à Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)

   à Duplicati.Library.Main.Controller.Backup(String[] inputsources, IFilter filter)

   à Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)

And here is the error message of the attempt to repair the database:

Feb 7, 2019 3:39 AM: Failed while executing "Repair" with id: 3
Duplicati.Library.Interface.UserInformationException: Recreated database has missing blocks and 3 broken filelists. Consider using "list-broken-files" and "purge-broken-files" to purge broken data from the remote store and the database.

   à Duplicati.Library.Main.Operation.RecreateDatabaseHandler.DoRun(LocalDatabase dbparent, Boolean updating, IFilter filter, NumberedFilterFilelistDelegate filelistfilter, BlockVolumePostProcessor blockprocessor)

   à Duplicati.Library.Main.Operation.RecreateDatabaseHandler.Run(String path, IFilter filter, NumberedFilterFilelistDelegate filelistfilter, BlockVolumePostProcessor blockprocessor)

   à Duplicati.Library.Main.Operation.RepairHandler.RunRepairLocal(IFilter filter)

   à Duplicati.Library.Main.Operation.RepairHandler.Run(IFilter filter)

   à Duplicati.Library.Main.Controller.RunAction[T](T result, String[]& paths, IFilter& filter, Action`1 method)

   à Duplicati.Library.Main.Controller.Repair(IFilter filter)

   à Duplicati.Server.Runner.Run(IRunnerData data, Boolean fromQueue)

Looking at remote destination, I found 9 “dlist” files:

2019-02-13_09-33-11

That’s exactly what I did last time the backup crashed … But, is it reasonable to have to waste so much time trying to get things right? And to have to start all over again when it crashes and finally give up and recreate a backup from scratch?

I find it extremely damaging not to be able to rely on a backup system. Duplicati is clearly not up to its ambitions and it can in no way be described as a reliable backup system.

Thank you to all three @Pectojin @johnvk and @ts678, but I realize during our discussions that finally, once this crashes, there is not much left to do … It’s time for me to find another solution other than Duplicati


#11

Sorry we couldn’t get things going for you. Part of the reason Duplicati is still in beta is because, while it works well for many users, when it doesn’t work well it’s a pain to deal with.

Good luck with your future backups! If you find something you think is comparable to Duplicati, feel free to let us know.

And of course, please consider Duplicati again when we (eventually) get to a stable release. :slight_smile:


#12

I hear the concerns about the instability. At this point, they are valid.

However, I am sticking with Duplicati for the time being for the following reasons. These are just my opinions (subject to change):

  1. Acronis is heavy on resources
  2. Paragon is hard for (non-tech) users to understand (so if I’m going to be helping them / setting it up anyway…)
  3. Carbonite slowed down 1 system I investigated, and does not include large files nor video files by default
  4. BackBlaze Backup doesnt keep enough versions
  5. Duplicati is open source
    • and standards based, meaning, you could (with difficulty) extract your backups from the .dblock files by hand, without Duplicati, if you had to. Ie, no lock-in.
  6. Duplicati is free
  7. (Perhaps since Duplicati is free) it is focused on giving the user maximum choice
  8. Duplicati via hashing ensures integrity of backup files–especially important with online backups
    • This hashing complicates Duplicati and is responsible for a lot of the bugs.
    • File-sync programs that compare only dates and file sizes and store files as plain files in the backup are not as complicated, have fewer bugs, and the backup files are directly viewable/usable/recoverable. Those are significant advantages. However, if there’s corruption, you’re out of luck. I can’t depend on that for long-term, widespread usage.
  9. Duplicati is compatible with an enormous range of platforms. It’s quite remarkable.
  10. Duplicati had 204 votes on alternatives.to - highest of the free options (that’s what brought me here in the first place)
  11. Duplicati has a very active forum and vigorous user community for help and work arounds
  12. Duplicati is in active development with a capable team
  13. The lead developer @kenkendk posts on the forum regularly.

Duplicati looks to me to be on track to be an excellent software program, and I’m excited to be in on the ground floor.


#13

Thank you… Sure, I will :slight_smile:


#14

@johnvk

I agree with your arguments … But the balance is still in favor of other solutions, because Duplicati is not sufficiently operational and especially because I can not rely on him to have a safe backup…

Thank you for your support…