What do i have to do? To study the problem
I’m not sure what level you want to study at. Code is probably here, though I don’t know it, and exactly how pause and resume fit into it isn’t clear, so it might take some further research into Duplicati design.
If you want to study it from its external behavior, then see if you can get exact small steps to reproduce this, so that any developer could get it to double reliably on any system. You can see how I was setting certain scenarios with a trivial local folder backup, and watching the clock carefully (at least in minutes).
If you can actually get it to do four of the same backup in a row, that would be nice. Haven’t seen that…
You wanted none at all? That will take some careful schedule planning and setting of schedule.
The design is (I believe) to run missed backups when it becomes possible to do that, however
if you wanted no double-backups, that’s the (presumed) issue that we’re trying to investigate…
Yes that’s right you understand me.
I do not have extremely important data to do double backups after clicking the Resume button.
I need duplicati just to help me restore the config files for programs or my personal data.
My data that I backup does not change often, so I need a double backup mode and I would like to turn it off or wait for a new duplicati update
Hello,
Well, it does. I’ve set pause to 10m and after system wake up (hibernation):
22 Apr 2020 11:33 - Operation: Backup
Ok, that was one “missed” backup, but the next one:
Next scheduled task: (…) Today at 11:44
So, there’s no double run this time, but the pause value was added somehow to the next schedule instance… So… let’s try set pause to 45m (in case of scheduler is some what confusing about hours values…). Turn off computer, and…
22 Apr 2020 14:00 - Operation: Backup.
That "missed’ one from my schedule, when computer was off.
But I got next:
Next scheduled task: (…) Today at 14:38
Now, that number seems to be out of blue, correct one should be 14:45 (e.g pause time) or 15:27 (e.g 14:00 + 87m of my schedule timer). But after looking at my logs, it seems that Duplicati just walk through schedule - the last one was at 11:45, then I turn off computer, turn on at 13:15, wait 45m of the pause, 14:00 run “missed” one, then scheduled one is at 14:38 because: 11:45+87m+87m=14:38 (voilà!).
Possibly still not understanding. Possibly a translation issue?
So you don’t want double, but you also say you “want none at all” on Resume. How does that fit:
then this flips. Is there a missing “do not” before “need” below? That then means you want single.
I suspect that this is a bug (not sure), if you are suggesting that these double are done as a mode.
Turn what off? Again, I suspect this is not a designed mode, so there’s no “switch” to turn mode off.
There may be workarounds, which are still being looked for. Removing or lowering pause may help.
Possibly early Resume button use works differently than letting the pause interval run out by itself?
It appears that @ZebCorp is testing without Resume button, but your original issue used Resume.
My early tests were all manual Pause and Resume. An overnight sleep-wake-auto-Resume did OK.
You can turn off the schedule for any backup on job’s screen 4 Schedule. Uncheck this, then Save:
Eventually this will likely be resolved (more help will help), but any fix will go in Canary release first.
These are very new, and not for everybody (new code can have new bugs). Beta are less frequent.
Feel free to clarify your request, but Duplicati cannot be enhanced for every possible thing desired.
If you don’t clarify, I assume you want the back-to-back double backup stopped, but do want single.
11:45 - 87 minutes is 11:44, so maybe 11:44 was on schedule, and 10 minutes after 11:33 by chance.
My expectation of what it should do is exactly what the schedule page says for next time and interval.
I also see it explains the missed-backup policy at the top. I think I first heard of that from forum posts:
For testing purposes, perhaps an even schedule like 1 hour would reduce some required date math.
For whatever it’s worth, my overnight test had a 5 minute pause setting, and did one run each of two scheduled jobs, 5 minutes after the first Windows System event log time of system wake in morning.
After those missed jobs ran, both jobs next ran at their appointed schedule time. I didn’t hit Resume.
my log have 3 copies of backup 2 of them superfluous happened today after
from sleep
In the settings I have is to do backups every 185 minutes and this time 3:00
“If a date was missed, the job will run as soon as possible.” applies to the first one because your 185-minutes apart schedule didn’t happen overnight because your system was sleeping. It wakes, it runs.
So this looks on the surface like “missed job” catchup plus scheduled backup then unwanted second.
Well here you see the problem? Or is it operating status duplicati
This is presumably a problem. It will take some time and effort and skill to investigate. Have patience.
These issues might be related:
I became acquainted with the themes of githab, I have exactly the same problem.
Only different situations but it is similar to that described by other contributors.
I’m really looking forward to the correction and improvement of functional
Thanks for the findings!
The first one sounds sounds very much like what I get on my Pause/Resume misses, however it needs at least two misses – I suspect GitHub issue got at least two due to 15 minute schedule, but am unsure.
Answering the question there about multiple misses queuing more, it doesn’t for me, and doesn’t clearly happen in this forum issue, assuming the first one is a catch-up of the overnight miss, which I assumed.
One subtle point, though, that I got into in my tests was whether a miss during sleep and a miss during Duplicati Pause with the system not sleeping work the same way. My test needed sleep during pause…
I’m not quite sure where @ZebCorp test wound up. It would be nice to see what is reproducible, which is why I was lobbying for a clear and complete step-by-step recipe that anybody could try themselves…
Second GitHub issue sounds like a request to be able to disable the currently always-run catch-up run. Getting super-fancy would look into how close the run was to next, but this is all enhancement options.
said earlier is still how this looks. Though that test used manual Pause/Resume button press, people were using Settings Pause after startup or hibernation so I tested that Pause with Resume at expected time by Settings, and also manual Resume just before and just after second missed backup.
Test setup was sort of covered earlier, but to recap, here are new steps:
-
Create backup job of a short file (one word of text), no encryption, to local folder destination,
Advanced options–upload-unchanged-backups set, scheduled every 10 minutes at the 10 minute mark simply to make the operation and interpretation a little bit easier mathematically. -
Sleep and wake Windows before scheduled run, so that Settings Pause will miss one or two.
-
Optionally you can watch About → System info at page bottom and also page top status box.
-
Optionally you can watch About → Show log → Live → Information to see the backups run.
-
Open Restore dropdown when things finish to see whether it did a single backup or a double.
I’m thinking through this and earlier testing that it’s Paused-while-Windows-is-awake that does this, meaning a workaround is to make sure the Pause can’t cause two backup misses, meaning for 185 minute interval between backups, a 3 hour Pause should go OK. There might have been some info contrary to that, but I’m having trouble finding it, so if anybody sees otherwise, please be very clear.
Anyone’s welcome to do their own tests to find out what works or doesn’t. My test results are below:
Pause 15 minutes with automatic Resume
7:17 sleep Windows
7:18 Windows awake
7:20 missed backup 1
7:20:54 schedulerQueueIds adds miss
7:30 missed backup 2
7:33 backup run 1 as expected
7:33 backup run 2 as an issue
Pause 5 minutes with automatic Resume
7:47 sleep Windows
7:48 Windows awake
7:50 missed backup 1
7:51:10 schedulerQueueIds adds miss (and status box "Next task" stops saying missed time)
7:53 backup runs once only
Pause 15 minutes with manual Resume just before second miss
7:58 sleep Windows
7:59 Windows awake
8:00 missed backup 1 (didn't catch time when it noticed)
8:09:55 manual Resume
8:10 backup runs once only
Pause 15 minutes with manual Resume just after second miss
8:18 sleep Windows
8:19 Windows awake
8:20 missed backup 1
8:21:16 schedulerQueueIds adds miss (and status box "Next task" stops saying missed time)
8:30 missed backup 2
8:30:05 manual Resume
8:30:05 backup run 1 as expected
8:30:06 backup run 2 as an issue
so it still seems to take two misses, but way of Resume (automatic timer, or manual) doesn’t matter.
Way of Pause (earlier test used button, and this round of tests used Settings Pause) doesn’t matter.
Hello @ts678,
My humble 2 cents.
Test (1)
Pasue 10m
Wake: 11:31
Next task: 11:40
System info:
estimatedPauseEnd : 2020-04-23T 11:41:17.870567+02:00
proposedSchedule : 2020-04-23T 09:40:00Z
11:41 - Operation: Backup
11:41 - Operation: Backup
wake: 11:53
next task: 12:00
System info:
estimatedPauseEnd : 2020-04-23T 12:03:05.8319498+02:00
proposedSchedule : 2020-04-23T 10:00:00Z
12:03 - Operation: Backup
12:03 - Operation: Backup
Test (2):
Pasue 45m
Wake: 11:01
Next task: 12:23
System info:
estimatedPauseEnd : 2020-04-23T 11:46:13.0339704+02:00
proposedSchedule : 2020-04-23T 10:23:00Z,
11:47 - Operation: Backup (single)
Next task today at 12:23
So, just a single thought, maybe…
if proposedSchedule>estimatedPauseEnd (e.g test 2) then no problem else double run (e.g test 1)?
The bad news is it looks like the two misses that lead to the duplication can include the missed run.
Sitting overnight is very likely to get a missed run, so after it’s awake and paused, one more does it.
Pause 10 minutes with automatic Resume. Backup interval 10 minutes as previous.
2:38 sleep
2:40 missed backup 1
2:42 wake
2:44:10 stops talking about next task at 2:40
2:50 missed backup 2
2:54:35 backup run 1 as expected
2:54:36 backup run 2 as an issue
Thanks for help with trying to characterize this. Perhaps it will help a developer volunteer who digs in.
EDIT:
I updated the GitHub issue on this as well, saying that they could come here to get some testing data.
Perhaps this will help find the cause, but regardless I see it as helpful to be a little less of a mystery…
It also seems (in tests so far) to suggest that a pause leads to an issue, so one workaround is clear…
1 Like
I can replicate this, and I think I know the cause of the problem. I’ll have to investigate more carefully to come up with a candidate fix.
The following fixes the simple case that I was testing with. Can someone else (perhaps @ts678, @drwtsn32) run some additional tests to verify?
1 Like
Nice work, I’ll test it out tomorrow…