Stuck in Verify backend data..

1

Good morning everyone,
I tried to search for information on the forum regarding my problem, but I found very old topics, so I thought it would be useful to reopen the discussion.
I have Duplicati installed in a Docker container and have performed the first backup of my datastore.
The datastore is about 4TB. The backup is done on an external disk mounted on Linux.
I noticed that Duplicati performs the backup, but it has been stuck on “Verify backend data” for 2 days.

Regarding permissions, the container is run with Root permissions, and it can write to the “target” backup directory without any issues. (In fact, it has currently created numerous files as usual.)

Yesterday, I checked the system activities in htop and did not notice any resource usage like CPU/RAM/disk by Duplicati (which was happening throughout the entire backup process in the past few days).
I think the backup might have been completed and it possibly got stuck in some final verification step.

If i go to the “Information” menù and then “System Info” this are the “system propriety” :

lastEventId : 18533
lastDataUpdateId : 11
lastNotificationUpdateId : 3
estimatedPauseEnd : 0001-01-01T00:00:00
activeTask : {“Item1”:2,“Item2”:“1”}
programState : Running
lastErrorMessage :
connectionState : connected
xsfrerror : false
connectionAttemptTimer : 0
failedConnectionAttempts : 0
lastPgEvent : {“BackupID”:“1”,“TaskID”:2,“BackendAction”:“List”,“BackendPath”:null,“BackendFileSize”:-1,“BackendFileProgress”:0,“BackendSpeed”:-1,“BackendIsBlocking”:false,“CurrentFilename”:null,“CurrentFilesize”:0,“CurrentFileoffset”:0,“CurrentFilecomplete”:false,“Phase”:“Backup_PostBackupVerify”,“OverallProgress”:0,“ProcessedFileCount”:91180,“ProcessedFileSize”:4061802638031,“TotalFileCount”:91180,“TotalFileSize”:4061802638031,“StillCounting”:false}
updaterState : Waiting
updatedVersion : 2.0.8.1_beta_2024-05-07
updateReady : false
updateDownloadProgress : 0
proposedSchedule :
schedulerQueueIds :
pauseTimeRemain : 0

What do you all think?
Thank you in advance.!!

2

Welcome to the forum @zewol

It sounds quite strange, especially without any CPU use. Make sure you’re looking closely enough.
Some processors have a lot of cores, and depending on the tool, one busy core is little in the total.

You can see if About → Show log → Live → Profiling shows anything, but it’s not meant for history.

I see “BackendAction”:“List” posted above, and wonder if it finished. You can see if it will let you see
<job> → Show log → Remote, and if there’s a list line, clicking on it will expand into the file listing.
After that, there’s probably a lot of SQL, which can be slow on large backups, but it uses resources.

3

Hi and thanks for the response!
This morning I opened the web interface and the backup was completed.

Last night, however, I noticed that in the system information, the “lastEventId” field was continuing, so it was doing something.

For future occasions, can I rely on that counter, or is there another system to understand if it’s stuck? It would be great if something written appeared on the main bar.

This is the log screen.

" {“Size”:52373886,“Hash”:“b3wXRq3X/1ADALGMpzLAqvwf7M4PD17J+OGnqEtpEBs=”} "

LOG
LOG1170×2532 222 KB

Thank you very much!

4

Duplicati has many parts. From a look at the code (an expert dev could say better), I think that’s polling. Typically there’s a long poll every 5 minutes or so from anything needing status updates from Duplicati.

What is HTTP Long Polling ?

Verifying backend files

At the end of each backup job, Duplicati checks the integrity by downloading a few files from the backend.

The TEST command

A sample consists of 1 dlist, 1 dindex, 1 dblock.

… except if configured differently, but backup-test-samples defaults to 1 sample, and is what it did.

Before that is the list command I was asking about, and asking you to click on. You can do it now.
Seeing two is not normal. Possibly the second is a retry that apparently happened the same minute.
Logging to a log-file can show in seconds. That and About → Show log → Live can detail any Retry.

2024-05-31 05:48:59 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started:  ()
2024-05-31 05:48:59 -04 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed:  (4 bytes)

shows how you get explicit Started and Completed times. The message of “bytes” should say “files”.

This verification runs both before and after the backup, so you’ll have lots of chances to see it further.

which seems like it should be consistent (compared to a network), yet there was a possible list retry awaiting your close look to see what those two list actually got, and maybe adding logs if it continues.