Sorry in advance for the lengthy post but just want to make sure I’m setting the software up correctly for what I want it to do.
Background: I’m a Photographer and am migrating from Crashplan as they have changed maximum retention times for deleted files to a max of 90 days. I have Duplicati installed on a server running Win Server 2012 R2. Am using Backblaze B2 for remote cloud storage with Duplicati (also use Windows default backup tool for locals to external drives that I swap out etc). All my work files live on the server.
I have one master “editing queue” folder where I import all new images to work on & edit them. Once work is completed the raw image files and rejects (large size, can be 60/70gb for a wedding) are deleted and just the finished jpegs are kept (generally 5/6gb).
Currently (until the policy changed) I had Crashplan set to keep the most up to date version of deleted files for 1 year for this folder. Am I right in thinking that the Duplicati Smart Backup Retention option should do basically the same? If not what retention policy would keep the latest version for 1yr after deletion from my local files?
Secondly, I have an “Archive” where I move the finished jpegs too. I had Crashplan set to keep the most recent file version for this folder indefinitely even if deleted locally. I have not had to delete locally yet but as space on my server is becoming a premium I may start deleting older jobs/file but would like these to remain indefinitely in the cloud backup. I have Duplicati to set to “Keep a specific number of backups” and the number is 1. Again am I right in thinking this will keep the most recent version of a file indefinitely.
Note on both the above I’m not too fussed on versioning; files would rarely if ever change
Lastly, is there any way of getting a notification emailed or otherwise if a backup job fails, or do I just have to keep an eye on it myself?
Not exactly. CrashPlan had a special retention option for deleted items, but Duplicati doesn’t that feature. Instead Duplicati keeps the entire backup version for a certain amount of time (depending on your retention policy). It never considers individual files within a backup job separately when it comes to retention.
The Smart Retention policy might be fine as it keeps versions for up to 1 year. But note it doesn’t keep ALL backup versions for that 1 year. For the first week, only one backup is retained per day. Beyond that, for the first 4 weeks it only keeps one backup per week. And beyond that, for the most recent 12 months it only keeps 1 backup per month. So if a file only existed on your system for 1 week and then was deleted, it may not be recoverable after the first month depending on exactly when it was deleted. Instead of Smart Retention you can craft a custom retention option that errs on the side of safety more, if that’s your desire.
Nope. As mentioned above Duplicati is talking about entire backup versions, not individual files. There isn’t a way to tell Duplicati to keep 1 version of each file forever. Telling Duplicati to keep only 1 backup version means that when you complete a backup job it will delete the older backup versions. So if a file gets deleted, and you do another backup, that file will become unrecoverable.
Duplicati isn’t really an archive system, it’s a backup system. I personally don’t recommend you delete any source data as then your only copy is in Duplicati and it’s no longer a “backup.”
You can use its integrated email notification feature, or you can use a 3rd party service like duplicati-monitoring.com.
Crashplan was a very good software in the beginning and very bad at the end for me.
My daily archive at the end (I left CP few months ago) would take 6-8 hours with crash plan and the escalation engineer told me it is normal and it is my fault - not CP version 8 fault (so they just closed my case).
Duplicati achieves the same for about 20 min.
Since you are a photographer - it is possible to organize the backup in many different ways. For example the “working” files can be set to delete the old versions entirely after shorter period of time while the “done” files can be set to keep the old versions longer.
Duplicati can be specific on the timing. 24h is 24h - not 23h or 25h (so try to be overprotective for important backups - that are recent - like “unlimited” for a week).
Since Backblaze charges on download - the files associated with the editing (.xmp for me - I am using Darktable) can be kept locally (a second backup) - this is because if retrieval is needed - it would save you from paying for downloads from B2.
Also - on B2 it is not needed to keep the old versions - just current. This is because duplicati actually creates its own versions.
At the end - I find duplciati much nicer than Crashplan - my current experience is positive. And there is a lot of help here.
2 Likes