After successfully backing up my data to an external hard drive I ran the backup again to test it and see how long it took to do incrementally. All went well at a speed of 10 -12 Mb/sec till it got to the last 2.34 Gb when it suddenly slowed to around 8Kb/sec and has been crawling ever since. The following logs appeared when I paused the backup for 5 minutes. I am working in Windows 10.
.
Hi @Clare welcome to the forum!
From the progress bar, it looks like it is still en the process of scanning files.
The error messages are related to queries done by the UI so you can ignore those.
If you use the “Live” log area and choose “Profiling” as the level, it may reveal what exactly is being processed and why this causes a slowdown.
Thank you. I had a look at the Live profile and the only thing I could see that looked relevant was the following:
My understanding of the background workings is pretty limited.
How much data? Has the incremental taken longer already? If not, there might be other factors.
Incremental backups
Duplicati performs a full backup initially. Afterwards, Duplicati updates the initial backup by adding the changed data only. That means, if only tiny parts of a huge file have changed, only those tiny parts are added to the backup. This saves time and space and the backup size usually grows slowly.
and it also means that upload speeds (which are an average) can slow due to little to upload. Looking at the job area on the home screen will show files being read through. Is it constant? Seeing it sitting at the end of a file for awhile probably means it’s scanning for more changes.
Because it’s an average, sudden slowdown seems unlikely. A slow decline seems more likely.
You can look at that to see if files are going out. Or look at About → Show log → Live → Retry.
Yes the original backup of 369Gb took nearly 11 hours but the incremental one has been going now for over 24 hours and appears stuck at one point. No files are being read through at all - the same file remains on screen all the time. See attached jpeg.
It looks like Duplicati is backing up itself, which explains some of the slowness.
The file it is stuck on in the screenshot is the database that Duplicati uses to track its work.
If you exclude that folder, it will work much better, otherwise you will see an exponential increase in storage+time.
Thanks for your help. I went into Edit to see where I could exclude that file as you suggested, and hit the Test button to check the connection to my external hard drive but it has now hung on that - whereas before it was quick to verify the connection. Just says Testing… . I am wondering if I shouldn’t start from scratch and format the external drive - or move to a different product?
has more discussion on what might be seen here, especially if Database screen confirms that the file being read is related (ignore its -journal suffix) to the job that’s currently running. There’s no benefit to backing up one’s own database, as it’s always stale because it changes during backup.
except when it can’t change, in which case Duplicati code that needs a database might stop then, resulting in a low-activity hang-type situation. Are status bar “to go” count and size declining any?
Task Manager can also show if it’s using CPU and other resources. If not, I’m not sure if TrayIcon Quit will work in hung situation. See if Task Manager shows it going away. If not, kill it and restart.
This will probably fix Test connection, and exclude should avoid colliding with active database.
Thank you. Advice much appreciated.