Progress bar "to go" numbers incorrect [2.0.3.8 & 2.0.3.9]

Since updating to 2.0.3.8 (and then 2.0.3.9) my backups are running better (yay!) but my progress bar is much worse. :flushed:

Specifically, a test job that I know has 171MB across 7 files shows a progress bar of this (where itā€™s ALWAYS set to 2 files):
image

Is anybody else (@drakar2007?) seeing this as well or have a just managed to screw up my own computer again? :slight_smile:

For as quickly as this goes away on my work laptop backup job (since not a whole lot is added each time), mine appears after a quick glance to be running right. But Iā€™ll pay attention to it more closely in the coming days to make sure Iā€™m right.

Thanks. Honestly Iā€™ve only been noticing that it happens when setting up test jobs so theyā€™re mostly initial runs thus take longer. (And maybe thatā€™s why Iā€™m seeing it in the first place.)

Iā€™ve seen this same issue when manually running jobs, I have 2.0.3.9. Although really I never paid any attention to that since it has never really been correct anyways since my first testing with 2.0.3.2 until the latest 2.0.3.9.

Similar in my case!
No green progress bar on top, but if finishing the backup and verifying the files the green progress bar goes to 90% and finally 100%, no progress in between.
The ā€œ2 filesā€ from your screenshot are 4 in my case, constantly, no matter how may files to backup. Maybe something with the parallel threads / processes which take care of the files to backup and encrypt?
I have set the parameter --paralell-somithing-I-do-not-remeber to 4 if I remember correctly. Havenā€™t checked yet what happens if I change the parallel threads to any other number than 4. Will do it next weekend.

Looked at the source code now for several days :confused: but have not figured out how to fix this. :tired_face:

Thanks for the additional confirmations!

This happens to me even if I purposefully hobble --concurrency-block-hashers=1 (default), --concurrency-compressors=1 (default 1), and --concurrency-max-threads=1 (default 0).

I havenā€™t tried checking yet but I wonder how far back youā€™d have to revert to get the bar working againā€¦that might help in identifying what part of the code needs to be fixed now.

I have become a pro in the meantime by changing back and forth the different commits :confused:
The green progress bar worked until commit v.2.0.3.5-2.0.3.5_canary_2018-04-13 (version bump) three month ago! A lot of great parallel processing came into play.
Then Kenneth fixed it partially with commit v2.0.3.7-2.0.3.7_canary_2018-06-17 because I reported a strange behavior with the lastPgEvent but the main bar stays a little weardā€¦

Even rolling forward each commit did not get me closer to any solutionā€¦ may it help someone else hopefully :grinning:

1 Like

FWIW, this is still happening on 2.0.4.23_beta_2019-07-14. Mine is showing this:

0 files (-269148 bytes) to go at 432.55 KB/s

The byte count was already negative when there were 960 files to go.

LMK if I can help debugging this.

Hi @Zieren,

There are a couple known reasons for this. One is due to a bug where a file can be momentarily counted twice against the ā€œtotal bytes to processā€ number. This can send the ā€œbytes remainingā€ number negative. Mostly noticeable on small backup sets where one file is a very large percentage of the total. This particular bug has been fixed in the Canary channel but hasnā€™t made it into the Beta releases yet.

The other known instance of it going negative is when files grow during backup. At the start of a backup job, files are enumerated and Duplicati gets an idea of ā€œtotal bytes to process.ā€ If a file has grown by the time Duplicati gets around to backing it up, it can throw the ā€œbytes remainingā€ into negative territory. Filesystem snapshots are one way to work around this problem.

If you saw it go negative and stay negative when there were still 960 files to go, Iā€™m guessing it might be the second scenario in your case. Can you confirm that you are not using snapshots? Are you running on Windows?