Restore newest is 2 weeks old; later backups empty

Support
#1

Today I tried to restore a file. I was shocked that the “Latest” shows as 0:Feb 7 2020 6:42 pm. even though my backups have been running daily and no recent errors have been shown.

That time of the latest restore corresponds to when I had to repair/delete/rebuild the local database due to an error being thrown by the backup before then. I can’t tell you the error message since logs prior to the repair have disappeared, but it was something about blocks (missing?). At that time, I searched the forum and found instructions that said to repair or delete/rebuild the database, which I did.

However, now looking at all of the recent backups, they all seem to be empty! This is very perplexing!

Help!

#2

Do you mean empty or missing? It sounds like Restore had nothing newer than 0:Feb 7 2020 6:42 pm, which makes me wonder how you’re looking at all the recent backups. Are there logs? What’s in them? What release is this? 2.0.5.1 Beta has a nice display of source file changes. No changes, no backup…
–upload-unchanged-backups=true can override that. It’d be nice if that’s all it is. Basically, need data…

#3

Empty. You are currently running Duplicati - 2.0.5.1_beta_2020-01-18

Every log summary shows this:
Source Files Examined 0 (0 bytes) Opened 0 (0 bytes) Added 0 (0 bytes) Modified 0 (0 bytes) Deleted 0

Here is an example of a complete log:
> {
> “DeletedFiles”: 0,
> “DeletedFolders”: 0,
> “ModifiedFiles”: 0,
> “ExaminedFiles”: 0,
> “OpenedFiles”: 0,
> “AddedFiles”: 0,
> “SizeOfModifiedFiles”: 0,
> “SizeOfAddedFiles”: 0,
> “SizeOfExaminedFiles”: 0,
> “SizeOfOpenedFiles”: 0,
> “NotProcessedFiles”: 0,
> “AddedFolders”: 0,
> “TooLargeFiles”: 0,
> “FilesWithError”: 0,
> “ModifiedFolders”: 0,
> “ModifiedSymlinks”: 0,
> “AddedSymlinks”: 0,
> “DeletedSymlinks”: 0,
> “PartialBackup”: false,
> “Dryrun”: false,
> “MainOperation”: “Backup”,
> “CompactResults”: null,
> “VacuumResults”: null,
> “DeleteResults”: null,
> “RepairResults”: null,
> “TestResults”: {
> “MainOperation”: “Test”,
> “VerificationsActualLength”: 3,
> “Verifications”: [
> {
> “Key”: “duplicati-20191212T200000Z.dlist.zip.aes”,
> “Value”:
> },
> {
> “Key”: “duplicati-i0559c9dc8bf74702b38d93ff88dd3d1f.dindex.zip.aes”,
> “Value”:
> },
> {
> “Key”: “duplicati-b2891eb3e1a2f4ee49e7fe92a65a841e6.dblock.zip.aes”,
> “Value”:
> }
> ],
> “ParsedResult”: “Success”,
> “Version”: “2.0.5.1 (2.0.5.1_beta_2020-01-18)”,
> “EndTime”: “2020-02-08T06:41:03.039549Z”,
> “BeginTime”: “2020-02-08T06:40:50.640297Z”,
> “Duration”: “00:00:12.3992520”,
> “MessagesActualLength”: 0,
> “WarningsActualLength”: 0,
> “ErrorsActualLength”: 0,
> “Messages”: null,
> “Warnings”: null,
> “Errors”: null,
> “BackendStatistics”: {
> “RemoteCalls”: 5,
> “BytesUploaded”: 0,
> “BytesDownloaded”: 18748135,
> “FilesUploaded”: 0,
> “FilesDownloaded”: 3,
> “FilesDeleted”: 0,
> “FoldersCreated”: 0,
> “RetryAttempts”: 0,
> “UnknownFileSize”: 0,
> “UnknownFileCount”: 0,
> “KnownFileCount”: 32945,
> “KnownFileSize”: 172075001837,
> “LastBackupDate”: “2020-02-07T18:42:58-05:00”,
> “BackupListCount”: 233,
> “TotalQuotaSpace”: 0,
> “FreeQuotaSpace”: 0,
> “AssignedQuotaSpace”: -1,
> “ReportedQuotaError”: false,
> “ReportedQuotaWarning”: false,
> “MainOperation”: “Backup”,
> “ParsedResult”: “Success”,
> “Version”: “2.0.5.1 (2.0.5.1_beta_2020-01-18)”,
> “EndTime”: “0001-01-01T00:00:00”,
> “BeginTime”: “2020-02-08T06:37:00.217126Z”,
> “Duration”: “00:00:00”,
> “MessagesActualLength”: 0,
> “WarningsActualLength”: 0,
> “ErrorsActualLength”: 0,
> “Messages”: null,
> “Warnings”: null,
> “Errors”: null
> }
> },
> “ParsedResult”: “Success”,
> “Version”: “2.0.5.1 (2.0.5.1_beta_2020-01-18)”,
> “EndTime”: “2020-02-08T06:41:03.164292Z”,
> “BeginTime”: “2020-02-08T06:37:00.217123Z”,
> “Duration”: “00:04:02.9471690”,
> “MessagesActualLength”: 11,
> “WarningsActualLength”: 0,
> “ErrorsActualLength”: 0,
> “Messages”: [
> “2020-02-08 01:37:00 -05 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Backup has started”,
> “2020-02-08 01:38:46 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()”,
> “2020-02-08 01:39:47 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (32.17 KB)”,
> “2020-02-08 01:39:49 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()”,
> “2020-02-08 01:40:49 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (32.17 KB)”,
> “2020-02-08 01:40:50 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20191212T200000Z.dlist.zip.aes (7.93 MB)”,
> “2020-02-08 01:40:56 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20191212T200000Z.dlist.zip.aes (7.93 MB)”,
> “2020-02-08 01:40:56 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-i0559c9dc8bf74702b38d93ff88dd3d1f.dindex.zip.aes (4.26 KB)”,
> “2020-02-08 01:40:56 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-i0559c9dc8bf74702b38d93ff88dd3d1f.dindex.zip.aes (4.26 KB)”,
> “2020-02-08 01:40:56 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-b2891eb3e1a2f4ee49e7fe92a65a841e6.dblock.zip.aes (9.95 MB)”,
> “2020-02-08 01:41:02 -05 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-b2891eb3e1a2f4ee49e7fe92a65a841e6.dblock.zip.aes (9.95 MB)”
> ],
> “Warnings”: ,
> “Errors”: ,
> “BackendStatistics”: {
> “RemoteCalls”: 5,
> “BytesUploaded”: 0,
> “BytesDownloaded”: 18748135,
> “FilesUploaded”: 0,
> “FilesDownloaded”: 3,
> “FilesDeleted”: 0,
> “FoldersCreated”: 0,
> “RetryAttempts”: 0,
> “UnknownFileSize”: 0,
> “UnknownFileCount”: 0,
> “KnownFileCount”: 32945,
> “KnownFileSize”: 172075001837,
> “LastBackupDate”: “2020-02-07T18:42:58-05:00”,
> “BackupListCount”: 233,
> “TotalQuotaSpace”: 0,
> “FreeQuotaSpace”: 0,
> “AssignedQuotaSpace”: -1,
> “ReportedQuotaError”: false,
> “ReportedQuotaWarning”: false,
> “MainOperation”: “Backup”,
> “ParsedResult”: “Success”,
> “Version”: “2.0.5.1 (2.0.5.1_beta_2020-01-18)”,
> “EndTime”: “0001-01-01T00:00:00”,
> “BeginTime”: “2020-02-08T06:37:00.217126Z”,
> “Duration”: “00:00:00”,
> “MessagesActualLength”: 0,
> “WarningsActualLength”: 0,
> “ErrorsActualLength”: 0,
> “Messages”: null,
> “Warnings”: null,
> “Errors”: null
> }
> }

#4

How do you inspect empty new backups? The Restore dropdown shows 0 at the top like this, right?

image

0:Feb 7 2020 6:42 pm was said to be the latest. Do you mean there are new ones on dropdown?

How can one choose newer than 0 on the restore selector, and does “empty” refer to folder/file box? Can you pull up any files if you look for them in the Search for files box, then hit Search button?

Still trying to understand what’s seen where. That log does look odd though, examining no files at all.
Is Last successful backup on home screen increasing daily? What about log “BackupListCount”?

You could try watching live log at About --> Show log --> Live --> Verbose during backup to see if any source file names are mentioned. If you’re running –usn-policy in Advanced options, you can try off.

Please also verify that screen 3 Source Data still specifies existing folders that you want backed up.
Commandline would also show chosen source folders and files in the Commandline arguments box.
This would also be a good place to review options, just in case there’s something there that’s related.

Direct inspection of the destination files can also show what the new backups are doing, or not doing. There should be a set of files with names with date and dlist in their names. Are new ones smaller? Possibly they’re missing (back to missing versus empty confusion). The log you show uploaded none:

“FilesUploaded”: 0,

#5

No, the Dropdown does show the Latest being two weeks ago, Feb. 7.

image
image1418×705 89.7 KB

The list of log files show every day since then, with no errors returned. However, if I open any of those logs, the show all zeroes. There are definitely files changed each and every day, since this machine is my daily driver.

image
image1965×1282 179 KB

#6

Thanks for clarifying what’s what. The troubleshooting steps given earlier are still a reasonable path.

Is source all files on a local drive, or might some system issue be making them unseen? If you start creating a new backup job (don’t actually Save or Run), how does the view of things look right now?

It looks like the Feb 7, 2020 6:42 PM Repair got a Warning (yellow icon). Anything interesting there?

#7

I think I found the solution, and it looks like a Stupid User Trick. When I was looking through the live log, I noticed the Filters. I decided to inspect the configuration, since I knew that I had to edit the filters. I noticed the Exclude Expression * shown in the attached. I have no idea how that got in there, but since I edited the Filters to add the Duplicati reference, it must have been me. Now I am running the backup, and it is doing many files (seems like a full initial backup), so I think everything is working OK.

Thanks for the help

image
image974×1106 86.8 KB

1 Like
#8

You can possibly figure out when things went wrong by looking at older versions on Restore dropdown. Possibly something changed after the Feb 2, 2020 3:00 PM backup when dailies stopped being made?

What “should” happen is that files should be read, and it should be determined that their blocks exist in uploaded backup files already from last time files were present, and it should just reference them again. That’s its block-based deduplication at work. Processing similar data gives some of the technical detail.
You can look at your backup’s “FilesUploaded” or “BytesUploaded” to see what actually got uploaded…

I’m glad you solved the main mystery.