Backup for every weekday on a dedicated hard disk (HDD)

Support
1

I plan to back up every day of the week in the evening to a dedicated hard disk. The hard disks each have their own name (e.g. Monday, Tuesday, etc.). Every morning the hard disk of the previous day is replaced by the current one, except when I am on vacation.
The actual used backup solution created further backups on the hard disk connected during the vacation. If the correct test disk was reconnected after the vacation, the backups were not made immediately, but only in the evening.

I also have a monthly hard disk that I connect sporadically every month.

I have already searched the Internet for possibilities and have found the following two options:

  • Create a backup rule for each day.
  • Create a backup rule and write the database to the hard disk.

The disadvantage of both solutions for me is that a missed backup is carried out immediately after a hard disk is connected (e.g. after a vacation).

What other advantages and disadvantages do the two solutions have?
Is there a possibility that missed backups are not made up immediately, but only for the next planned period?

The backup also includes an Oracle database!
Is anything to be considered here? Can the backup data be corrupted if the database is written at the same time?

2

This is confusing me a bit. The backups will either run or fail, there is no option to delay that backup if the destination is not available. Do you have any scripts or similar that is preventing Duplicati from progressing?

Is this with an external scheduler? There is no support for this in Duplicati. How do you inform the system that there is a vacation?

The safe way to make a backup of a database is to either create a backup file from within the DBMS and then back that up. Otherwise, issue a flush of the database and then use a disk snapshot.

If you are on Windows, most DBMS solutions integrate with VSS, so making a VSS snapshot will also flush the database. For Linux you need some scripting and LVM to make it work.

3

I am currently using a different backup tool other than Duplicati. The backup tool I currently use does not distinguish between the different hard disks (for each day). When it finds a hard disk, it makes a backup. When I am currently on vacation, a backup will be run on the connected hard drive every day.

I believed I had read that, but unfortunately I can no longer find the source.

Duplicati should run on a Windows computer. I have no access to the database to perform a flush. The program that uses the database is also limited here.

If you could give me further links or support for creating a database backup, it would be helpful.

4

Duplicati is very picky about being connected to the right storage destination. It will not work if you change the disks, as it will detect that the disk does not contain the correct files.

I cannot think of a great way to set it up to work with rotating disks. You could have 7 different backups and then run each on their respective day and use a marker file to check if it is the correct backup. But this will create a bunch of warnings.

If you have an external scheduler, you can perhaps leave a marker file on each disk that ties the disk to the backup. Then you can use ServerUtil (or the regular CLI) to trigger the right backup. It is still a bit messy because you would need to have one backup configuration per disk.

Another, perhaps better solution, would be to edit the single backup job to keep the local database on the removable drive. This would make the metadata shown in the UI incorrect, but would then correctly make a new backup from the last state. This requires that the path to the drive is stable, so you can have the same path despite the rotating disks.

Then you need to use VSS snapshots, which requires running Duplicati with elevated privileges (usually “Run as Administrator”).

5

I have found the text on which I base my statement that missed backups will be made up as soon as the external hard disk is accessible again!

When setting up the scheduled backup, the following can be read “If a date was missing, the job will run as soon as possible”

Bildschirmfoto_20241226_190553
Bildschirmfoto_20241226_1905531082Ă—595 103 KB

I interpret the note that if a backup is not possible due to the absence of the external USB hard disk, the backup will be made as soon as the hard disk is available again.

6

Duplicati is a general-purpose backup program, with general-purpose messages and docs.

The word “possible” there likely means many things. Examples may be:

  • It’s been started
  • Computer is not sleeping. Interestingly, Windows 3 A.M. update check doesn’t wake it.
  • It’s not paused
  • It’s not busy running another job (a frequent question is run order, if there is a backlog)

The word “run” is likely equivalent to pressing the “Run now” button. Doesn’t guarantee now, however it’s not a scheduled thing. In either case, the run may be delayed, or might not work.

One can set up jobs to bad drive letters, one can suffer network Source or Destination down. Problems like these are detected during the job run, and will likely give an error or a warning.

Avoiding such environmental problems can be done with a script that runs before the backup.

Example Scripts (notice the power, and the ability to steer run continuation with an exit code).

Windows Drive Letters might give some relief, as it can tie a job to a specific drive plugged in, regardless of what drive letter Windows gave it. It also links to a claimed path for stable letter.

A downside of, for example, launching seven jobs and having only one succeed is noise IIRC.

Rolling your own scheme in a script can make as much or as little noise as you like about this.

One exotic scheme that I haven’t tested (maybe your other respondent knows if it can work) is leveraging that drive letter detection in File backend to let script set database drive letter using documented ability to change destination on the fly. The question is whether run order allows it.

Drawing from an old test to see what environment variables were around for script, I see these:

DUPLICATI__REMOTEURL=file://C:\Duplicati\duplicati-2.0.8.1_beta_2024-05-07\RUN\test 1\

DUPLICATI__dbpath=C:\Duplicati\duplicati-2.0.8.1_beta_2024-05-07\RUN\SWLFIUQNCG.sqlite

So if drive letter probing sets target URL soon enough, script can use it to set DB path drive letter.

is what this scheme has in mind. I do that to a USB drive, but it does get busy with database work. Rotation isn’t what I’m after. I just wanted to see how it would work, and for self-contained backup.

1 Like