Setting up Duplicati only as a CLI on a web/db server

How-To
1

Hello

I installed Duplicati on Windows fine and now I am trying to setup Duplicati in pure CLI mode (no desktop) on web servers to backup to Backblaze B2 Cloud.

I blogged about my setup here Backing up files to a Backblaze B2 Cloud Bucket with Duplicati

I ran “systemctl enable duplicati” and “systemctl start duplicati”

It says it is up :slight_smile:

"systemctl status duplicati

  • duplicati.service - Duplicati web-server
    Loaded: loaded (/etc/systemd/system/duplicati.service; enabled; vendor preset:
    Active: active (running) since Fri 2019-06-07 21:06:53 AEST; 57s ago
    Main PID: 17247 (mono)
    Tasks: 14 (limit: 4915)
    CGroup: /system.slice/duplicati.service
    |-17247 DuplicatiServer /usr/lib/duplicati/Duplicati.Server.exe
    `-17251 /usr/bin/mono-sgen /usr/lib/duplicati/Duplicati.Server.exe
    "

Duplicati is ready

“which duplicati-cli
/usr/bin/duplicati-cli”

Q1) I am a bit confused as to how to schedule a backup to an existing B2 bucket daily with a smart backup retention? So many guides but which one to use?

Any advice woudld help.

2

Hi @feartec, welcome to the forum!

The CLI and Web-UI are not the same, and operate independently.

If you use the CLI, you need to use cron or something similar to run the backups periodically. You can use duplicati-cli help to get started. You do not need the server running for this.

For the web-UI we do not currently have any tools for manipulating the backups other than the main WebUI itself. You need to somehow forward/tunnel access to the server, and then access the WebUI from a local browser.

1 Like
3

Hey that was quick. :slight_smile:

So I should disable the service and instead move to cron?

e.g…

#crontab -e
…
#Run Duplicati Backups
6 2 * * * root -s /bin/bash -c ‘/scripts-run-duplicati-backup.sh’
…

4

I guess systemctl disable duplicati would do the trick?

For setting it up with the commandline, there is documentation here:

That would be the retention-policy:

If you have already set it up to your liking with the GUI, you can export the job and choose “As commandline” to get the exact string you would need to put into the cron job.

5

I have figured it out and updated Backing up files to a Backblaze B2 Cloud Bucket with Duplicati

Backup:
/usr/bin/duplicati-cli backup “b2://bucket-name-goes-here/test” “/test/” --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module=“zip” --dblock-size=“50mb” --passphrase=**********

Restore:
/usr/bin/duplicati-cli restore “b2://bucket-name-goes-here/test” “/test2/” --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --passphrase=************* --overwrite=true --restore-path="/test2"

Backup and email:
/usr/bin/duplicati-cli backup “b2://bucket-name-goes-here/test” “/test/” “/test2/” --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module=“zip” --dblock-size=“50mb” --passphrase=123 --send-mail-to=your@email.com --send-mail-subject=“Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)” --send-mail-body=“Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)” --send-mail-url=“tls://smtp.gmail.com:587” --send-mail-username=your@email.com --send-mail-password=*****************

Q1) How can I set smart backup retention on cli?

6

That is the --retention-policy option:

7

I think smart retention is a just a --retention-policy of 1W:1D,4W:1W,12M:1M to avoid doing a custom one.
It would be possible to see whether the above is correct by doing the suggested Export as Command-line.
The GUI description talk of “at least one remaining backup” is likely just –allow-full-removal=false (default).

8

Thanks, updated my post :slight_smile: Backing up files to a Backblaze B2 Cloud Bucket with Duplicati

9

Nice post. There doesn’t seem to be anything yet in the How-To category, but maybe you could fix that. :wink:

Two things caught my eye.

Backblaze B2 confuses lots of people ever since application keys got introduced. I think the rule is to pair Key ID with Application Key, or pair Account ID with Master Application Key, but post suggests the pairing:

* Enter your Backblaze Account ID ( *e.g* . ##########################)
* Enter the Buckey ApplicationKey ( *e.g* . ###############################)

Removed account id to reduce confusion #3647 is an attempt to simplify things, but it might over-simplify. There’s some talk there about what works.

Duplicati only offers full backups ( not incremental or differential backups ). This will be an issue if your backups are large.

doesn’t reflect what happens, which is actually closer to incremental-forever, except with back-references instead of a (potentially breakable) chain of additional increments. Every file, old or new, is a set of blocks, and existing blocks can be used again in new files if the content is the same. That’s the deduplication part.

Taking a new “full” backup actually uploads only the changed blocks, and everything else just points to old. This makes what seems to be a full backup, with the backup overhead of an incremental. The inefficiency of having to rebuild restore files from potentially various bundles of blocks can slow down restores though.

Block-based storage engine is an easier overview.
How the backup process works is more technical.
Choosing sizes in Duplicati is done to fine-tune the algorithm to the use. Large backups sometimes suffer from too many blocks to track, too many little files on the remote, and so on. Some of this can be tuned…

1 Like