Backup hangs at the end of Backup_ProcessingFiles

ts678 · January 5, 2021, 7:21pm

Isn’t there a timestamp and some other similar lines near it? Does timestamp change? Is next line just like this one except with different Hash? Possibly you’re seeing the following code, but one line doesn’t prove it.

github.com

duplicati/duplicati/blob/338f467111f6bdcde971f93b8376daba42827498/Duplicati/Library/Main/Operation/Backup/SpillCollectorProcess.cs#L117-L117


      
          await database.MoveBlockToVolumeAsync(file.Key, len, source.BlockVolume.VolumeID, target.BlockVolume.VolumeID).ConfigureAwait(false);

github.com

duplicati/duplicati/blob/338f467111f6bdcde971f93b8376daba42827498/Duplicati/Library/Main/Database/LocalBackupDatabase.cs#L823-L832


      
          public void MoveBlockToVolume(string blockkey, long size, long sourcevolumeid, long targetvolumeid, System.Data.IDbTransaction transaction)
          {
              using (var cmd = m_connection.CreateCommand())
              {
                  cmd.Transaction = transaction;
                  var c = cmd.ExecuteNonQuery(@"UPDATE ""Block"" SET ""VolumeID"" = ? WHERE ""Hash"" = ? AND ""Size"" = ? AND ""VolumeID"" = ? ", targetvolumeid, blockkey, size, sourcevolumeid);
                  if (c != 1)
                      throw new Exception(string.Format("Failed to move block {0}:{1} from volume {2}, count: {3}", blockkey, size, sourcevolumeid, c));
              }
          }

Success is good. I have no feel for what the times should be for 43 gb/30.000 files, given your hardware, network connection, backup history, and amount of change between backups. What does job log show?

<job> → Show log can give a summary of Source Files. The Modified line might be most relevant.
Complete log → BackendStatistics gives BytesUploaded and FilesUploaded. Is it 2 hours worth?

If you watch About → Show log → Live → Retry, you can see the uploads happening, and judge speed. Especially on non-initial backup, it can take a little time to gather enough changes to start uploading data. After that, it becomes a question of whether Duplicati can prepare faster than it can upload or vice versa.

Uploading that stops for some reason can also stop preparation with asynchronous-upload-limit in queue.