Hello everyone!
First I would like to say that Duplicati is absolutely awesome! I have yet to find anything that works closer to what I think how backups should work!
… Sadly that doesn’t mean I am 100% happy. ^^
First I’ll try to explain my situation and what I/we have done so far.
After that I have listed how I have set up Duplicati and a RAM-Disk for caching, so if you are just interested in that, skip ahead (to the “Duplicati”-Section).
I would be happy if anyone has suggestions on how to improve stuff for non-experts.
EDIT: This post got quite long as I tried to explain where I come from. Maybe there is some value to it as a kind off “new user experience” or something like that.
===============
Pre-Duplicati
I don’t have a lot of experience in backing stuff up, so I’m sure there is plenty to criticize.
Also I want to preface that we can’t spend a lot of money on this. Let’s just say that 30€ for THE perfect program would be the limit. So I would like to keep it free-ish … small payments for really good stuff are fine (including maybe donations for Duplicati?)
My Girlfriend has a lot of personal data. Stuff for work, huge amounts of music, pictures and videos from when she was dancing professionally. On top of that large amounts of raw audio from her recording hobby.
All in all that is approx. 3+ TiB of stuff that I don’t want her to lose.
Until now our solution has been a kind of manual Raid 1? That is, using FreeFileSync occasionally to spread duplicates over all her internal HDDs. But that wasn’t very robust and pretty tedious.
I have an old QNAP-TS210 2-bay-NAS that she could use.
It is to slow to work from it directly but I think it makes for an okay backup destination.
But there is no way the backup runs through in a single sitting. And as I saw when I tried Macrium Reflect Free Edition for my personal computer: even differential backups can take preeeetty long and if the connection is a bit flaky, everything is back to zero.
So I searched for something that:
- is free or cheap
- can deal with slow connections
- can quietly resume backups after interrupted connections or PC-restarts
- can deal with large datasets
- can do incremental backups to make regular backups in this setting feasible
- can consolidate the backup chain because our budget and drive space is very limited and there is no space for backup rotations
- is easy enough to use for her not to be totally scared about touching anything
- restoring backups is easy enough so that she can restore files when I am not around (which can sometimes be months)
So my first try was Macrium Reflect Free Edition. But the differentials were bigger and slower than I liked and there is no consolidation.
My second choice fell on Aomei Backupper as it can do incrementals and is user friendly … but we realized very quickly that the program is very unstable … at least on her system. Backups can freez after a couple hundred GiB and sometimes it seemed to break the the explorer (including taskbar!) so much that a hard-reset was required. Also when Aomei encountered any problems we weren’t able to fully terminate it which meant that the system would hang mid-shutdown, requiring a hard-reset as well.
Then I tried Veeam Backup & Replication Community Edition and I thought that this program was an absolute overkill as it seems to be targeted at servers and backing up large numbers of VMs. Nevertheless setting up a backup was mostly straight forward except that I couldn’t get it to actually do the backup. From what I read in their forums it’s some problem on local machines when admin accounts try to backup themselves or other admin accounts? The solution seemed to be some registry modification regarding UAC or something like that. But my girlfriend said that I am not going to modify the registry to sneak around some account protection thingy and I accepted that, so Veeam had to go … which it didn’t fully because apparently a clean install of Veeam is incapable of uninstalling itself without errors.
Then I discovered HardLinkBackup and loved the concept! Unfortunately the free version is very limited. I might have been tempted to pay for it, but unfortunately it cannot resume backups. The initial run has to be in one sitting. Which is a shame because having continuously incremental, compressed backups and being able to restore files just by using the file explorer is a massive plus for usability for inexperienced users!
Backup Service Home 3 seems to be very similar but way more complete than the free HardLinkBackup. Unfortunately it also cannot resume backups. Probably worth checking out for everyone who needs very simple backups on smaller datasets!
================
Duplicati
Other than doing system-images (I’ll probably use Macrium for that) Duplicati seems to do everything I want!
…
With a few exceptions.
Her PC has a small OCZ-SSD as the system drive and I feel pretty uncomfortable writing multiple terabytes through its temp directory when there is more than enough RAM available for caching the volumes.
So this is how I set up everything for a test on my system:
-
install Duplicati and change “Duplicati.GUI.TrayIcon.exe” to always start as admin
-
install ImDisk
-
copy “create_RAM_Disk.bat” and “destroy_RAM_Disk.bat” (see below) to the Duplicati installation directory (they require Duplicati to be started as admin)
-
deactivate autostart for Duplicati in the task manager (because it can’t start Duplicati as admin?)
-
set up task scheduler to launch Duplicati with elevated privileges on log-in
-
open Duplicati web-interface and paste my “standard settings” (see below)
-
set up backup plan including “–alternate-destination-marker” and “–alternate-target-paths” for more robust external HDD detection on my system.
(By the way, these options seem to get deleted every time the backup-configuration-panel is opened in the web-interface. Each change requires re-applying them.)
===================================
create_RAM_Disk.bat
imdisk -a -t vm -s 4G -p “/fs:ntfs /q /y” -m Z:
label Z: RAM_Disk
===================================
destroy_RAM_Disk.bat
imdisk -D -m Z:
===================================
Standard Settings
–asynchronous-upload-folder=Z:\Temp
–asynchronous-concurrent-upload-limit=10
–blocksize=200KB
–use-block-cache=true
–concurrency-compressors=4
–dblock-size=100MB
–hardlink-policy=All
–snapshot-policy=Auto
–synchronous-upload=false
–thread-priority=belownormal
–usn-policy=Auto
–run-script-before-required=C:\Program Files\Duplicati 2\create_RAM_Disk.bat
–run-script-after=C:\Program Files\Duplicati 2\destroy_RAM_Disk.bat
=====================
Final Thoughts
Now, I think for non-tech people this is quite the setup process and I really wonder if there is a simpler way to achieve what I want?
Different steps, other programs?
(I am aware that there are many things that I could improve by spending money. Better SSD, so I don’t need to care about caching. More HDD-space for RAIDs and other backup paradigms. Better NAS. Proper, paid software … although that would have been a total fail if I had bought the full version of Aomei Backupper as it completely disintegrated itself after a couple days of use.)
(I am also aware that some of the problems I had may be down to specific issues with her system … but honestly … every Windows installation develops some kind of weird issues after some time. I expect any backup software to be robust enough to not explode for unknown reasons when the system works fine otherwise. I want my girlfriend to install a backup-program, not to re-install EVERYTHING (without even having proper backups).)
That said, I am super happy that I found Duplicati and I hope the project stays healthy and progresses quickly.
Big THANK YOU to everyone who even read through half of all of this!
And sorry if my english sounds funny or if I wrote something stupid. I’m not native to the language nor to the topic. ^^
PS: Is “Support” the correct category for this post?