# Backup order after "missed" time schedule

**URL:** <https://forum.duplicati.com/t/backup-order-after-missed-time-schedule/20626>\
**Category:** Support\
**Created:** [May 5, 2025, 8:05pm UTC](https://forum.duplicati.com/t/backup-order-after-missed-time-schedule/20626 "2025-05-05T20:05:22Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![asman](https://forum.duplicati.com/letter_avatar_proxy/v4/letter/a/ba8739/32.png) [@asman](https://forum.duplicati.com/u/asman)\
**Post date:** [May 5, 2025, 8:05pm UTC](https://forum.duplicati.com/t/backup-order-after-missed-time-schedule/20626/1 "2025-05-05T20:05:22Z")

</div>

I have scheduled my nightly backups in a fixed order. I need the order to be maintained, because the “last” backup has to run last. It has a run-script-after hook that shuts down the duplicati container and unmounts the backup drive.

The jobs are ordered by their time schedule in the GUI. This works like a charm in normal operation. However, when duplicati misses a backup run (cause host was shut down for some days) the backup run order seems to be random. That means, the “last” backup runs too early and shuts down the container before the other jobs are done.

Are there ideas how to improve that? I am open for a better “shutdown trigger”, I just need a way to find out if all scheduled jobs are finished.

As a short summary to give an idea:  
0:30: cron on the host system: backup destination drive (extern USB drive) is switched on and mounted, duplicati docker container is started  
0:31 - 0:35 : several “real” backup jobs are scheduled in duplicati  
(they may take longer, which is no problem because duplicati does not run them in parallel anyhow)  
0:40 .. the “last” backup job is scheduled (it may be started later, this is OK as long as it comes last). After completion it triggers a script on the host, that shuts down the duplicati docker container and unmounts / switches off the backup destination drive

The approaches I have already discarded:

- Put everything into one backup job: I want different retention policies for different source folders. Afaik it is not possible to have different retention settings in the same backup job.
- Trigger the jobs via CLI: I like the GUI for defining the jobs and seeing the logs.

---

<div class="post-metadata">

**Author:** ![ts678](https://forum.duplicati.com/letter_avatar_proxy/v4/letter/t/8491ac/32.png) [@ts678](https://forum.duplicati.com/u/ts678)\
**Post date:** [May 6, 2025, 8:22pm UTC](https://forum.duplicati.com/t/backup-order-after-missed-time-schedule/20626/2 "2025-05-06T20:22:48Z")

</div>

> [@asman](#):
>
> 0:31 - 0:35 : several “real” backup jobs are scheduled in duplicati

Does this really mean that several permanently-daily-scheduled jobs run after cron start, however system downtime causes missed schedules, and some missed jobs get in first?

> [@asman](#):
>
> the backup run order seems to be random

> [@In what order does missed backup jobs run?](https://forum.duplicati.com/t/in-what-order-does-missed-backup-jobs-run/6329/1):
>
> But it seems they are started in the order they happen to be shown on the web page

was an earlier theory, however it’s not certain. So you’re sure yours is looking random?

> [@Reordering jobs](https://forum.duplicati.com/t/reordering-jobs/6633/4):
>
> > [@Reordering jobs](https://forum.duplicati.com/t/reordering-jobs/6633/3):
> >
> > Is it possible they are running in the order they were scheduled?
> 
> Possible, though I’d like to hear what actually happens. It looks like the scheduler gets its list [here](https://github.com/duplicati/duplicati/blob/79d839546b238be79967bf8532267bfab2b08aa6/Duplicati/Server/Scheduler.cs#L227) which does a SELECT [here](https://github.com/duplicati/duplicati/blob/79d839546b238be79967bf8532267bfab2b08aa6/Duplicati/Server/Database/Connection.cs#L721) which (per SQL rules) comes out in no particular order because no ordering was requested, however if the database decides to use [primary key](https://github.com/duplicati/duplicati/blob/79d839546b238be79967bf8532267bfab2b08aa6/Duplicati/Server/Database/Database%20schema/Schema.sql#L30) order, then maybe it’s whatever ran and rescheduled first.

> [@asman](#):
>
> Trigger the jobs via CLI: I like the GUI for defining the jobs and seeing the logs.

They are not mutually exclusive. Duplicati 2.0 could start GUI jobs from CLI using:

[A command line client for controlling the Duplicati Server](https://github.com/Pectojin/duplicati-client) (third party contribution)

[ServerUtil](https://docs.duplicati.com/duplicati-programs/command-line-interface-cli-1/serverutil) is for Duplicati 2.1, and has more in it than the documentation mentions:

```plaintext
Commands:
  pause <duration> Pauses the server []
  resume Resumes the server
  list-backups List all backups
  run <backup> Runs a backup
  login Logs in to the server
  change-password <new-password> Changes the server password
  logout Logs out of the server
  import <file> <passphrase> Import a backup configuration
  export <backups> Export a backup configuration
  health Checks the server health endpoint
  issue-forever-token Issues a long-lived access token
  status Gets the server status

```

One approach (if the “missed” run order is truly random – which maybe dev can resolve) would be to have cron start Duplicati as current, and either run the first backup or let it be permanently scheduled. First job has run-script-after to run the second, etc. Force chain.

---

<div class="post-metadata">

**Author:** ![asman](https://forum.duplicati.com/letter_avatar_proxy/v4/letter/a/ba8739/32.png) [@asman](https://forum.duplicati.com/u/asman)\
**Post date:** [May 7, 2025, 12:36am UTC](https://forum.duplicati.com/t/backup-order-after-missed-time-schedule/20626/3 "2025-05-07T00:36:49Z")

</div>

> Does this really mean that several permanently-daily-scheduled jobs run after cron start, however system downtime causes missed schedules, and some missed jobs get in first?

all jobs are missed during the downtime.

> So you’re sure yours is looking random?

Let’s say they are not run in order of the schedule times.  
I would expect the job with the most “overtime” to be run first, which does not happen.

> [ServerUtil](https://docs.duplicati.com/duplicati-programs/command-line-interface-cli-1/serverutil) is for Duplicati 2.1

Thanks for the hint, I will look at this once I have updated.

---

<div class="post-metadata">

**Author:** ![ts678](https://forum.duplicati.com/letter_avatar_proxy/v4/letter/t/8491ac/32.png) [@ts678](https://forum.duplicati.com/u/ts678)\
**Post date:** [May 7, 2025, 12:48am UTC](https://forum.duplicati.com/t/backup-order-after-missed-time-schedule/20626/4 "2025-05-07T00:48:42Z")

</div>

> [@asman](#):
>
> all jobs are missed during the downtime.

Of course, but I’m trying to understand your configuration. Is it a daily schedule, meaning when cron starts Duplicati at :30, those missed jobs manage to run before :31 - :35 jobs?

> [@asman](#):
>
> I would expect the job with the most “overtime” to be run first, which does not happen.

OK, so old theory of run in creation order (top-to-bottom on screen) might still be correct, and if so you can control run if you’re willing to export/import last job to bottom of screen.

New Duplicati Canary test versions have a sorting feature that will throw people off if they have found that screen order is run order of missed jobs, but that may or may not be true.

---

<div class="post-metadata">

**Author:** ![kenkendk](https://forum.duplicati.com/user_avatar/forum.duplicati.com/kenkendk/32/5305_2.png) [@kenkendk](https://forum.duplicati.com/u/kenkendk)\
**Post date:** [May 9, 2025, 1:28pm UTC](https://forum.duplicati.com/t/backup-order-after-missed-time-schedule/20626/5 "2025-05-09T13:28:58Z")

</div>

> [@ts678](#):
>
> New Duplicati Canary test versions have a sorting feature that will throw people off if they have found that screen order is run order of missed jobs, but that may or may not be true.

The sort order is display only; it has no effect on the scheduled order.

> [@asman](#):
>
> Are there ideas how to improve that? I am open for a better “shutdown trigger”, I just need a way to find out if all scheduled jobs are finished.

The scheduler system in Duplicati is not really ready for anything other than simple requests.

If you need full control, you can use `duplicati-server-util` to start the backups when you need them to. You then configure all backups without a schedule and then externally (via cron or similar) you invoke `duplicati-server-util` to make backups. The scheduler in Duplicati is a queue, so you will get stable scheduling if you call them back-to-back.

There is a `--wait` option, but it is currently a bit broken, so you need to use the `list` command to see if any backups are running when you wait for the shutdown.

---

<div class="post-metadata">

**Author:** ![ts678](https://forum.duplicati.com/letter_avatar_proxy/v4/letter/t/8491ac/32.png) [@ts678](https://forum.duplicati.com/u/ts678)\
**Post date:** [May 9, 2025, 5:19pm UTC](https://forum.duplicati.com/t/backup-order-after-missed-time-schedule/20626/6 "2025-05-09T17:19:02Z")

</div>

> [@kenkendk](#):
>
> The scheduler in Duplicati is a queue, so you will get stable scheduling if you call them back-to-back.

If this means a FIFO (first in first out) queue in order of ServerUtil run, I’m not seeing that.

Make job called `test A` that waits a minute and backs up A.txt. Make similar for `test B`.  
Run a third somewhat long job to allow time to run both test jobs (into queue) as B then A.

ServerUtil status:

```plaintext
Active task: [Task 23]: BackupId = 1
Scheduled tasks:
  [Task 25]: BackupId = 5
  [Task 24]: BackupId = 6

```

Result per logs is that when the long ID 1 job finishes, test A (5) runs, then test B (6) runs.  
If “stable scheduling” means something other than “run” order, what order is queue run in?

Results here could be explained as BackupId order, but I don’t know if that’s by the design.  
If it’s by design (as opposed to SQLite choice), may it apply to the missed job runs as well?

---

<div class="post-metadata">

**Author:** ![kenkendk](https://forum.duplicati.com/user_avatar/forum.duplicati.com/kenkendk/32/5305_2.png) [@kenkendk](https://forum.duplicati.com/u/kenkendk)\
**Post date:** [May 12, 2025, 11:52am UTC](https://forum.duplicati.com/t/backup-order-after-missed-time-schedule/20626/7 "2025-05-12T11:52:33Z")

</div>

> [@ts678](#):
>
> If this means a FIFO (first in first out) queue in order of ServerUtil run, I’m not seeing that.
> 
> Make job called `test A` that waits a minute and backs up A.txt. Make similar for `test B`.  
> Run a third somewhat long job to allow time to run both test jobs (into queue) as B then A.

Nice that you tested it!

I only looked at the queue itself, which is very much FIFO based.

But, it turns out that the API to run a backup explicitly asks to override the queue, so it will insert the new task at the start of the queue.

I think the reason for this was that if the user request to run a backup task, it should be more important than the scheduled ones. It does however prevent a reliable scheduling from an external tool, and ServerUtil uses the API without any special considerations.

I have a [fix ready for this](https://github.com/duplicati/duplicati/pull/6247), where ServerUtil will be able to ask for next spot, but by default just append to the queue.