Then there is some detective work required. How did those files disappear?
Judging from the timestamp and the filenames, this looks like a single backup that has disappeared, specifically on from 2025-07-18, so not a new backup. A guess would be that this backup was deleted by the previous backup, but somehow the database update was not persisted. Alternative is that something else deleted the files, which does not sound plausible.
Do you have any way of figuring out where the files went? B2 logs? Duplicati logs?
Still valid. The idea is to detect stalls and otherwise don’t care how long the operation takes.
It is not stalled as I understand it. The problem is that we activated chunked uploads to B2, and it looks like this is handled incorrectly when used with B2. It looks like the transfers actually complete, but we do not get the transfer to complete when it is done, instead it waits for a response from B2 which can be delayed.
With the fix, the activity timeout should only cover the transfer itself.
That should work the other way around. There should be more files registered (the “failed” uploads), but this is usually fixed automatically.
Did you also see the problem with a higher timeout value?
I share the concern and would like to understand how to replicate the issue.
EDIT: Created an issue to track this.
Yes, that certainly looks to be the case. Is there any way to log this on the B2 side, so we know if B2 considers the file received or not? We rely on B2 to send a 200 - OK status code back, but maybe there is a deviation somewhere?
I would really like a look, maybe I can find a clue somewhere.
Yes, that is the rule.
Yes, that is the logic. Once the transfer starts (or really, when the file is ready to be transmitted), its status is recorded as “Uploading” in the database. This is committed to disk before the actual transfer is done to prevent “unknown” files from being found on the remote destination.
After the transfer completes, the status is updated to “Uploaded”, but this is a bit less strict in commit timing due to the asynchronous nature of the upload.
During the file list check, a file that is listed as “Uploading” but is found in its full length is promoted to “Uploaded” to account for the gap in transaction commit.
If the file is “Uploading” and missing or incomplete, it is removed, as it is assumed the transfer failed. If it exists, it is deleted as well. Some other code can sometimes create a replacement file (index files and filelists).
If the file is “Uploaded” and missing or incomplete, this is an error because the backend has acknowledged the transfer, but now the file is gone. And this is what @jberry02 reports happens.
Do you have the stack track for that one? We have discovered an error in the Test code that does this, and I think we have generally fixed it, but I would like to verify.
Do you have the exact string so I can edit it? I think it suggest “purge” because you could theoretically purge the files in question manually, but maybe purge-broken-files is a better suggestion.