Big Comparison - Borg vs Restic vs Arq 5 vs Duplicacy vs Duplicati

Comparison
1

Hey folks,

Have a bit of time on my hands (all COVID induced…) and trying out / comparing the different backup solutions. My initial findings below. I hope to make this a living resource so happy to do more testing based on feedback and include here. Do let me know any testing desired and I’ll see what I can do.

Background

Backup source is ~107k files totalling 460Gb of storage. Inside is:

  • photo/video collection (about 95% of the storage) - long tail here with most files being small (3-4Mb jpg) but with some largish videos making up the majority of the space
  • some other documents / other records

I currently use Duplicati / Arq5 as “production” backup (production in so far as I still experiment with them…!)

Setup Notes

Computer is desktop PC connected via wifi to gigabit pipe to the world. Wifi bandwidth to router is generally 30-50MB/s in favourable (large file bulk transfer) conditions.

Versions

Borg - no windows binary available; currently running through a docker container. (TO DO: update when WSL2 is out and stable)

Borg/Restic - very important disabled windows antivirus “real-team protection” for the backup folder/process. These linux port-ins do not set themselves up properly and without this the backup times can be 30 times worse. File modification is based on size; modification date and not inode. This is a change from default.

Duplicati - the backups have the option “backup-test-samples” set to zero. This means that after checking for updates, duplicati will not download one block and test for consistency. While this is not the default, as mentioned above Duplicati is my production backup and I have found this step adds an unneccesary slow down and have had it disabled for years

Results

The table is getting a bit wide due to request on different backends and if having diffculty reading, can download excel version here

https://f000.backblazeb2.com/file/backblaze-b2-public/Backup_Tool_Comparison.xlsx

Feature Comparison

Item Arq 5 Duplicati Duplicacy Restic Borg
Version 5.16 v2.0.5.106 1.30 0.9.6 1.1.5
Storage Backend Choices Ok Great Ok Great (via rclone) Poor
GUI Yes Yes Yes No No
Block Sizes Vary - Small 50 Mb Vary - Small Vary - Small 500 Mb
Dedup Yes Yes Yes Yes Yes
Dedup Rolling ** Yes No Yes Yes Yes
UI - Backup Progress Yes Yes Yes Yes Sort of
UI - Restore Progress Yes No Yes No No
Logging / Statistics Good Good Good No Manual
Price $50 Free $20 new + $10/yr Free Free

Synthetic Benchmark

Item Arq 5 Duplicati Duplicacy Restic Borg
Target HDD HDD HDD HDD HDD
Initial Backup 15h 9h24m 14h20m 18h30m 12h
Backup Size 438Gb 435Gb 437Gb 447Gb 437Gb
Update (+0) 40s 1m 4s 1m04s 1m31s
Update (+200mb) 40s 1m12s 16s 12s 3m5s
Update (+2.1Gb) 2m30s 3m25s 2m13s 3m47s 4m42s
Restore (1 file; 3.7Gb) 3m30s 1m20s 1m51s 4m10s 2m30s
Restore (800 files; 2.7Gb) 2m30s 1m45s 1m16s 3m41s 1m40s
Dedup Stress Efficiency (100 * 1Mb) -6% -10% 95% -5% 0%
Dedup Stress Efficiency (16 * 8Mb) 48% -2% 96% 63% 45%
Dedup Stress Efficiency (4 * 128Mb) 98% 24% 99% 99% 98%
Prune 2h50m 78s 7s 7m25s* 40s

Real World Benchmark - Onedrive

Item Arq Duplicati Duplicacy Restic
Target Onedrive Onedrive Onedrive Onedrive
Initial Backup N/A N/A 72h+* 36h
Backup Size 438Gb 438Gb 440Gb 450 Gb
Update (+0) 40s 3m30s 33s 1m34s
Update (+200mb) 4m30s 5m20s 2m32s 1m52s
Update (+2.1Gb) 30m30s 6m35s 23m6s 6m12s
Restore (1 file; 3.7Gb) 48m 3m28s 20m28s 57m
Restore (800 files; 2.7Gb) 7m30s 4m35s 15m23s 22m

Real World Benchmark - Pcloud Virtual

Item Arq Duplicati Borg
Target Pcloud Virtual Pcloud Virtual Pcloud Virtual
Initial Backup N/A N/A 4h54m
Backup Size 438Gb 438Gb 436Gb
Update (+0) 40s 3m40s 3m42s
Update (+200mb) 1m10s 4m23s 2m16s
Update (+2.1Gb) 6m20s 7m47s 3m54s
Restore (1 file; 3.7Gb) 1h10m 1m30s 17m
Restore (800 files; 2.7Gb) 20m 14m 13m43s
11 Likes
2

Hi there! Thanks for this, however, having different target backends for each is a bit misleading, as backing up to a local HDD is always going to be faster ( unless you have more than gigabit on upload ). Also, it would be great to have a single target cloud backend to further improve consistency.
Two more things that could be tested are incremental updates with some change and restore times.

1 Like
3

I like it… I wish I had kept all my notes from when I compared numerous backup products ~3 years ago.

4

Thanks @Kelly_Trinh. A couple questions…

  • Is it possible to indicate what versions of each application you’re using?
  • Do the Duplicati update times include the verification step?
5

Hi Thanzex, thanks for your thoughts.

Agree that comparability is going to be difficult. FYI - I am actually on a gigabit connection but the issue isn’t really bandwidth as the “little file” approach means that the link is unlikely to be saturated. The bigger issue is latency where local HDD will hose the other providers. Unfortunately, I have limited disk space and can’t keep a 400gb backup store for each backup provider and so this is unlikely to be revised.

Updates and restore timings - it is the production backup; so basically when I have more baby photos to backup then will update :innocent: :rofl: :innocent:. Restore I will tackle after a few rounds of updates - likely to be artificial induced restore so any suggestion on setup can look into…

6

Version added.

Well spotted on the update times - it actually doesn’t include that step. This is intentionally disabled as when I have used Duplicati as production backup I have found that this step really drags down the time to completion (a 20 second update can become 2 minutes). I didnt really feel much value so I had explicitly disable and forgot until you mention justnow.

I have added clarification note in OP about this.

7

Thank you for this @Kelly_Trinh - interesting.
I also use Cloudbacko, which uses Ahsay technology. Ahsay have been known in the SMB field for many years and have now absorbed Cloudbacko’s product.
I have been using Cloudbacko’s product since the early inception and whilst I am not using the latest v4 release yet, I’ve looked at the release notes and the new features it has, which are missing in v3, are promising.
They do a Free and various paid for versions and works across all OSs. I mention it in case you might like to try it out too.

8

Hey dude - thanks for the intro.

If you are using Cloudbacko; why are you here? Spying on the enemy??? :slight_smile: :slight_smile: :slight_smile:

I’ll give it a shot but probably lower priority right now - the battery of tests does take a while to do (and it is manual!) but for any strong solutions happy to look into…

9

I have been using both Duplicati and Cloudbacko together for just over 3 years now. I first heard about Duplicati v1 through Steve Gibson’s podcast, so I have known about Duplicati for a long time.
For the avoidance of doubt, I am not hear as an “advertiser” for Cloudbacko. I mentioned Cloudbacko because I thought that, like me, you were always on the lookout for Cloud backup programs and that is the reason why you did the comparison. So, please don’t worry about doing a Cloudbacko comparison for me, that was not my intention. Apologies.

In case you are interested, the reason why I use both Duplicati and Cloudbacko is simple (in my eyes!):
At the time, I struggled to get VSS working with Duplicati and Cloudbacko works with VSS out of the box (I realise now that you have to have Duplicati running as a service, but back then, there wasn’t the helpful guides written!). Secondly, when I had Cloudbacko backing up a large amount of data to Onedrive, it was fine for a while and then started having errors and I had to rebuild the backup a few times (a pain on my slow connection). After a bit of research on Cloudbacko’s forums I realised that Onedrive isn’t great at being backed up to so I changed to an SFTP server, but from what I remember, I still had problems, so I switched to using Duplicati. There have been many Cloudbacko program revisions and bugs worked out since this experience 3 years ago however.
So I keep Cloudbacko backing up a tiny subset of program data that I need VSS for and it has remained fine on SFTP. I had a need to recently trial Cloudbacko on another PC for 30 days and didn’t have any issues with it backing up to Azure.

Thank you for the comparison data that you have provided; I’ve only recently come across Restic and Arq.

10

Hey dude; no worries - in case smiley didn’t come through; I wasn’t so serious.

I dont think there is too much territorial fights here; so exploring others and particularly if there is technical reasons (like yours) is definitely worth hearing IMHO.

Arq you should be a bit careful of; the author released v6 recently and it was a bit disaster/disorganized. I only kept in comparison because like you I am running a parallel setup (Arq and Duplicati)

All the best with it all; Cloudbacko testing in progress (initial thought; first backup sure is slow hahah)

11

I finished the preliminary run through Cloudbacko (where storage backend is local HDD).

Overall, disappointing from a technical point of view; seems to not support deduplication (at all; even a straight copy of file adds to the storage used). UI is quite nice but they also do not allow scheduled backups on the ‘free tier’.

Perhaps a case of over aggressive segmentation - since they do a lot of rah rah on website about support for VMs they may have the deduplication for higher tiers (but I am not going to pay just to do some testing).

Given lack of real competitive advantage (and lack of space) - unfortunately won’t be doing further round on cloud storage backends.

Updated the result summary with stats as well if curious (link in OP)

12

Thank you very much indeed for both your word of warning on Arq 6 and for trying out Cloudbacko @Kelly_Trinh.
I found this Cloudbacko forum thread from 2016 that states that deduplication is not a feature (at least for file backup) and I see no evidence of deduplication mentioned anywhere on their website or in program release notes since 2016. In the absence of asking Cloudbacko directly, in case something has changed recently and they have not shouted about it, I think that one has to conclude, as per your findings, that Cloudbacko still does not feature deduplication, which I did not realise.

I really want to thank you again for the work that you have done, it is an enormous and timely help to me. For this other PC I have to backup, I’ve really been around the houses searching for/trying out various programs and become exasperated with it all.

13

Why are you using such an old version of duplicacy? The current version is 2.6.2…

1 Like
14

Welcome back tophee!

On the versioning, I am testing the web-edition and the latest at the time was 1.3.0 (1.4.0 came out about a month or so ago… )

15

Oh, I see, this is the version number for the web-ui. I wouldn’t put that as the version of the backup engine because the web-ui is just a wrapper around the actual backup engine (a.k.a. the CLI version) but that’s just me. I can see how the duplicacy versioning (as well as licensing) is somewhat confusing.

1 Like
16

Lovely stuff, thank you so much. Here’s one more option:


bupstash

I don’t have any other objections for Duplicati currently, except the reliability issues. Luckily those are quite rear, but as discussed those shouldn’t basically ever happen, unless there’s data corruption for some other underlying reason.

17

Background:

I’ve used restic in the past and just started to use Duplicacy because pruning requires full downloads/repacking/re-uploads of most of the content with a restic repo - which just isn’t feasible with a slower internet connection.

I was wondering if you could expand your test a little, if you find some time:

  • It would be interesting how much traffic a purge/update/restore uses, as most people are not blessed with a gigabit internet connection :smiley:
  • Do you run VMs? While backing up pictures and documents are interesting, it would be neat to see how backing up several VMs looks like. Especially if they are not cloned, but installed individually. The disks should deduplicate very well, if the operating system and applications are similar.
  • It would be nice if you could test how well two different computers can deduplicate their backups onto one storage. I have a couple of computers with similar user-data and similar operating systems and would like to create backups from them onto the same storage and keep the disk usage at a minimum.
18

What does the star mean here? The number is quite different to the one in the xlsx

19

Hi there - its been a while so stretching my memory but from what I can see/recall, the testing was done quite a whlie ago and at the time restic was using an older (much) less efficient pruning method. Just around the testing was being performed, a improved prune approach / restic binary was being tested — so you can see in the xlsx the comment (with the much better 10s timing) was on ‘test version’

Good news is that the improved method works well and has been part of mainline for ages and so use the xlsx fast figure for comparison

20

Btw. Restic added compression features, pack size options and lazy prune options. Those were the two key things preventing me from using Restic. Now I start to think that it’s on par with Duplicati. Especially when (and if) Restic works reliably, but Duplicati corrupts data.

Gotta test it bit more.