"please verify the sha256 hash"?

I created a new test backup and completed the initial backup yesterday. The second upload today ended with a warning:

Warnings: [
    remote file duplicati-20180131T190000Z.dlist.zip.aes is listed as Uploaded with size 0 but should be 2152573, please verify the sha256 hash "yYyPmO1k+UWLtqc3TgxQQfFUUhMABEmQVduImYEb6Bg="

I was unable to glean a solution to this from the other topic discussing the same warning, so I’m asking for suggestions here.

Can you check the file size on the destination? If it’s 2152573 then my guess is the file hadn’t finished being written to the destination when Duplicati then queried for the file size.

Since the file existed but was still open for initial writing, the destination probably reported it as size 0. My GUESS is that this file will be just fine with the next run.

However, this has been reported enough that I’d like to ask @kenkendk what action the error message is suggesting when it says to “verify the sha256 hash”. I’m also curious if it would make sense to put a “size was reported as 0, wait a few seconds and ask again” block to better handle slow destinations. (Assuming my above guess is correct.)

Is the destination local storage, or a SMB or CIFS share?

Perhaps there’s a hash of the dlist in the local database? If it was a dblock it would be easy, since those hashes are in the dlist, but here I’m not so sure.

Either way, surely this should be done by Duplicati on discovery of the error?

Indeed, it is exactly 2,152,573 bytes.

So basically Duplicati is tripping over its own feet, or something?

Unfortunately, things are only getting worse. Today (i.e. “next run”) I not only got

Warnings: [
    remote file duplicati-20180201T205916Z.dlist.zip.aes is listed as Uploaded with size 0 but should be 2153469, please verify the sha256 hash "EWJonEggIuViniRBGLzY4Su5zCdZ6f4+dTgOXVRSxzA="

(same as yesterday), but also

Fatal error
Duplicati.Library.Interface.UserInformationException: Found 123 files that are missing from the remote storage, please run repair
   at Duplicati.Library.Main.Operation.FilelistProcessor.VerifyRemoteList(BackendManager backend, Options options, LocalDatabase database, IBackendWriter log, String protectedfile)
   at Duplicati.Library.Main.Operation.BackupHandler.PreBackupVerify(BackendManager backend, String protectedfile)
   at Duplicati.Library.Main.Operation.BackupHandler.Run(String[] sources, IFilter filter)

I checked a couple of the missing files and, indeed, none of them is in the storage. But why? I’m actually starting to suspect pcloud to be the perpetrator. as they also seem to have lost some other files of mine. The only (but significant) difference between the two incidents is that duplicati is accessing pcloud directly via webdav while the loss with duplicacy occurred via pclouds own client. So for cases to be explained by a bug on pclouds side, it would have to be a pretty fat bug, covering both interfaces.

UPDATE: This is weird: after running repair (without any error), I went back to the logs but I can no longer find the message with the missing files. There is no fatal error on the entire log page! I don’t understand. Does Duplicacy modify the logs after it repaired the database? If not (which I assume), then my next guess would be that when I previsously got the red error message, clicking on “view” did not bring me to the same place as I am looking at now. But where?

Also, I suspect that those missing files somehow were not missing in the end because when I ran the backup after the repair completed, it went rather fast so that I suspect not many files were uploaded. In any case, it was too fast for it to upload all the 123 missing files.

Definitely sounds to me like a flaky destination or communication issue.

The missing log entry could be due to only so many messages being kept in the log table (I don’t know how many).