Backups not showing in "Restore files"

Support
#1

Hi all,

Last night I accidentally nuked a NAS drive while setting up a redundant local disk (not my finest moment). Luckily, I had been running Duplicati to backup to Backblaze B2.

When I first setup the disk (now wiped), I took a backup via my desktop machine earlier in 2020. After that I plugged it into my NAS (a Raspberry Pi 3B+) and it’s been backing up daily ever since.

When I login to the Duplicati web portal on my NAS, it says “Last successful backup: Yesterday at 6:01 PM”), but when I go to recover the files, the earliest entry I see under “Restore from” is February 26th, 2020.

Most of the data should be available by that time, but I’m pretty sure I’ve added more files since then.

Is there a way to get more insight into this? Why does it show a successful backup yesterday, but the latest available option to recover is from February?

Duplicati version: 2.0.4.23_beta_2019-07-14

Thanks so much,
rothman

#2

In case it helps, here’s the latest entry in the logs (from “Reporting -> Show log”):

I see this interesting line, but again, the Web UI shows a successful backup from yesterday:

LastBackupDate: 27/02/2020 07:00:01 (1582786801)

DeletedFiles: 0
DeletedFolders: 0
ModifiedFiles: 0
ExaminedFiles: 1358371
OpenedFiles: 0
AddedFiles: 0
SizeOfModifiedFiles: 0
SizeOfAddedFiles: 0
SizeOfExaminedFiles: 967686146471
SizeOfOpenedFiles: 0
NotProcessedFiles: 0
AddedFolders: 0
TooLargeFiles: 0
FilesWithError: 0
ModifiedFolders: 0
ModifiedSymlinks: 0
AddedSymlinks: 0
DeletedSymlinks: 0
PartialBackup: False
Dryrun: False
MainOperation: Backup
CompactResults:
    DeletedFileCount: 2
    DownloadedFileCount: 0
    UploadedFileCount: 0
    DeletedFileSize: 1722
    DownloadedFileSize: 0
    UploadedFileSize: 0
    Dryrun: False
    MainOperation: Compact
    ParsedResult: Success
    Version: 2.0.4.23 (2.0.4.23_beta_2019-07-14)
    EndTime: 20/09/2020 01:59:47 (1600563587)
    BeginTime: 20/09/2020 01:46:39 (1600562799)
    Duration: 00:13:07.6591500
    Messages: [
        2020-09-19 23:18:55 +01 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Backup has started,
        2020-09-19 23:41:10 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started:  (),
        2020-09-19 23:41:42 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed:  (32.16 KB),
        2020-09-20 01:11:55 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b97d63a81a89a4a59911a2bf94513c019.dblock.zip.aes (845 bytes),
        2020-09-20 01:21:59 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-b97d63a81a89a4a59911a2bf94513c019.dblock.zip.aes (845 bytes),
...
    ]
    Warnings: []
    Errors: []
    BackendStatistics:
        RemoteCalls: 9
        BytesUploaded: 1722
        BytesDownloaded: 144420455
        FilesUploaded: 2
        FilesDownloaded: 3
        FilesDeleted: 2
        FoldersCreated: 0
        RetryAttempts: 0
        UnknownFileSize: 0
        UnknownFileCount: 0
        KnownFileCount: 32929
        KnownFileSize: 862704633341
        LastBackupDate: 27/02/2020 07:00:01 (1582786801)
        BackupListCount: 6
        TotalQuotaSpace: 0
        FreeQuotaSpace: 0
        AssignedQuotaSpace: -1
        ReportedQuotaError: False
        ReportedQuotaWarning: False
        ParsedResult: Success
        Version: 2.0.4.23 (2.0.4.23_beta_2019-07-14)
        Messages: [
            2020-09-19 23:18:55 +01 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Backup has started,
            2020-09-19 23:41:10 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started:  (),
            2020-09-19 23:41:42 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed:  (32.16 KB),
            2020-09-20 01:11:55 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b97d63a81a89a4a59911a2bf94513c019.dblock.zip.aes (845 bytes),
            2020-09-20 01:21:59 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-b97d63a81a89a4a59911a2bf94513c019.dblock.zip.aes (845 bytes),
...
        ]
        Warnings: []
        Errors: []
DeleteResults:
    DeletedSets: []
    Dryrun: False
    MainOperation: Delete
    ParsedResult: Success
    Version: 2.0.4.23 (2.0.4.23_beta_2019-07-14)
    EndTime: 20/09/2020 01:59:47 (1600563587)
    BeginTime: 20/09/2020 01:25:49 (1600561549)
    Duration: 00:33:57.7985410
    Messages: [
        2020-09-19 23:18:55 +01 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Backup has started,
        2020-09-19 23:41:10 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started:  (),
        2020-09-19 23:41:42 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed:  (32.16 KB),
        2020-09-20 01:11:55 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b97d63a81a89a4a59911a2bf94513c019.dblock.zip.aes (845 bytes),
        2020-09-20 01:21:59 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-b97d63a81a89a4a59911a2bf94513c019.dblock.zip.aes (845 bytes),
...
    ]
    Warnings: []
    Errors: []
RepairResults: null
TestResults:
    MainOperation: Test
    Verifications: [
        Key: duplicati-20200227T070001Z.dlist.zip.aes
        Value: [],
        Key: duplicati-i195aaffe5c20400e9223d5e6aa9089bc.dindex.zip.aes
        Value: [],
        Key: duplicati-bf51b2e8920e84667a078055f5010ff27.dblock.zip.aes
        Value: []
    ]
    ParsedResult: Success
    Version: 2.0.4.23 (2.0.4.23_beta_2019-07-14)
    EndTime: 20/09/2020 02:01:46 (1600563706)
    BeginTime: 20/09/2020 02:00:35 (1600563635)
    Duration: 00:01:11.1434730
    Messages: [
        2020-09-19 23:18:55 +01 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Backup has started,
        2020-09-19 23:41:10 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started:  (),
        2020-09-19 23:41:42 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed:  (32.16 KB),
        2020-09-20 01:11:55 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b97d63a81a89a4a59911a2bf94513c019.dblock.zip.aes (845 bytes),
        2020-09-20 01:21:59 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-b97d63a81a89a4a59911a2bf94513c019.dblock.zip.aes (845 bytes),
...
    ]
    Warnings: []
    Errors: []
ParsedResult: Success
Version: 2.0.4.23 (2.0.4.23_beta_2019-07-14)
EndTime: 20/09/2020 02:01:46 (1600563706)
BeginTime: 19/09/2020 23:18:51 (1600553931)
Duration: 02:42:54.9174000
Messages: [
    2020-09-19 23:18:55 +01 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Backup has started,
    2020-09-19 23:41:10 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started:  (),
    2020-09-19 23:41:42 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed:  (32.16 KB),
    2020-09-20 01:11:55 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Started: duplicati-b97d63a81a89a4a59911a2bf94513c019.dblock.zip.aes (845 bytes),
    2020-09-20 01:21:59 +01 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Put - Completed: duplicati-b97d63a81a89a4a59911a2bf94513c019.dblock.zip.aes (845 bytes),
...
]
Warnings: []
Errors: []
#4

The B2 Application Key I used has the following capabilities:

deleteFiles, listBuckets, listFiles, readFiles, shareFiles, writeFiles
#5

I’m guessing you haven’t added anything since February. Duplicati discards a backup version if there are no changes.

The job results you posted show that there are zero deleted, added, or modified files. So that particular backup would not have been retained.

#6

Just finished the restore and my files are looking great. I’ll be making a donation cause Duplicati really saved me.

While I’m ecstatic to have my files, might I suggest that in the case you mentioned, the dropdown items could say “February 26th, 2020 to September 20th, 2020” or some similar indication? That’d make it clear that Duplicati successfully ran, but noticed that nothing changed and decided not to persist a duplicate backup. I think that’d definitely be comforting to see while restoring files (a potentially stressful operation) and make Duplicati feel a bit more reliable.

Is GitHub Issues the best place to report and discuss a feature like that?

#7

upload-unchanged-backups can be used to make the backup visible. There’s not a great space cost, because all of the backup data itself is deduplicated, i.e. new backup just points to the existing data…

This would write a dlist file to the backup to list the files (and point to the data), allowing retention of knowledge that a backup was done, even for Direct restore from backup files, or database Recreate.

Features are done sparingly these days (too few developer volunteers) so it’s best to use what exists.
There’s nothing I can think of in the database currently that directly gives you what you’re asking for…
There are also oddball cases like partial interrupted backups and synthetic backups (on next backup).

#8

That sounds great. Thank you!

Might that interfere with the smart backup retention policy? Is that feature considered stable?

Thanks.

#9

It should work with any retention policy, and retention will have more work to do (cleaning up versions that are the same). I actually thought of suggesting some retention policy, so I’m glad you are already thinking about it.

The feature is stable AFAIK, but it still confuses people who think it’s calendar time. It’s not, just time intervals.

#10

Sounds great. I’ll give it a shot. Thanks again!