Welcome to the forum, @Phlogi - and thanks for the great list!
I believe some of those are also included in the built in “checkbox” excludes, but there’s no harm in having them in twice.
Personally, I feel some of the expressions are broader than I’d use (I usually limit things to the Users folder) but it’s a really good starting point for people who want to have broad source folders.
I’m being lazy here, but is the first one for excluding some CrashPlan files?
I was switching from CrashPlan to duplicati, that’s why I added this - just in case. It might be helpful to others too.
Also I guess the default exclusion from windows is a good idea, however I still got errors for files from user applications that were in use. Not sure how this should be treated. I remember that CrashPlan had some smartness to also backup those files once they became available and not complaining about them.
An example file:
C:\Users\user\AppData\Local\Microsoft\Windows\WebCache\WebCacheV01.dat
In Duplicati, the solution is:
Run Duplicati Server with administrative privileges (or run it as a Windows service with an administrative or Local System account) and set --snapshot-policy to on or required.
This will make Duplicati create a VSS snapshot using AlphaVSS, making it possible to backup files that are in use by another process.
This list looks great, and I’m attempting to use it. But on a Windows 8 PC, running Duplicati Beta, it is showing “Counting files” for hours instead of under a minute without the filters. This is on an SSD, so disk access is about as fast as it can be.
Is anyone else seeing similar results with big filter lists?
I’ve got a filter list about half your length but with minute regular expression bits (lots of “or”) and noticed no difference between running as a service or the tray icon.
Is the speed the same for you with the test-filters command?
Can you post your list too? Also I noticed that counting files is really slow for me too now. So I have to review that. I guess using less regular expressions should help.
I’ll run the test-filters command as soon as I find time.
Here’s my stuff, but be warned I do a bunch of half-completed testing and haven’t cleaned it up - plus it’s a carry over from CrashPlan rules which I was trying to set up to handle both older (XP) and newer Windows layouts.
Odd - now that I look again, it’s many fewer lines than I recalled.
-[.:\\(Users|Documents and Settings)\\.*\\([^\\]*Cache[^\\]*|[^\\]*Thumbnails[^\\]*|[^\\]*Temp[^\\]*|NoBackup|_NoBackup)\\]
-[.:\\(Users|Documents and Settings)\\.*\\[^\\]*\.cache]
-[.:\\(Users|Documents and Settings)\\[^\\]*\\(Videos|Downloads|dwhelper)\\]
-[.:\\(Users|Documents and Settings)\\[^\\]*\\AppData\\LocalLow\\]
-[.:\\(Users|Documents and Settings)\\[^\\]*\\AppData\\Local\\(ConnectedDevicesPlatform|HP|CrashDumps|Temp|ClassicShell|Microsoft|Packages|ElevatedDiagnostics|Comms|TileDataLayer)\\]
-[.:\\(Users|Documents and Settings)\\[^\\]*\\AppData\\Local\\Google\\Chrome\\User Data\\Default\\(.*\\LOCK|data_reduction_proxy_leveldb|Extensions\\|Cache\\|Application Cache\\|GPUCache\\|Media cache\\|Current Tabs|Current Session|Lockfile)]
-[.:\\(Users|Documents and Settings)\\[^\\]*\\AppData\\Local\\Google\\Chrome\\User Data\\(PepperFlash\\|CertificateTransparency\\|lockfile|CrashpadMetrics.pma[^\\]*)]
-[.:\\(Users|Documents and Settings)\\[^\\]*\\AppData\\Local\\Mozilla\\Firefox\\Profiles\\[^\\]*\.default\\(cache2|safebrowsing|saved-telemetry-pings|datareporting|saved-telemetry-pings)\\]
-[.:\\(Users|Documents and Settings)\\[^\\]*\\AppData\\(Local|Roaming)\\Thunderbird\\(Mozilla Thunderbird|updates|Crash Reports.*)\\]
-[.:\\(Users|Documents and Settings)\\[^\\]*\\AppData\\(Local|Roaming)\\Duplicati\\(\.*\.sqlite*|control_dir.*\\)]
-[.:\\(Users|Documents and Settings)\\[^\\]*\\AppData\\Roaming\\Microsoft\\Windows\\Recent\\]
-[.:\\(Users|Documents and Settings)\\[^\\]*\\AppData\\Roaming\\Thunderbird\\Profiles\\[^\\]*\\.default.*\\.*\\[^\\]*\.(msf|lock)]
-[.:\\(Users|Documents and Settings)\\[^\\]*\\NTUSER.DAT]
-[.:\\(Users|Documents and Settings)\\[^\\]*\\AppData\\Roaming\\Mozilla\\.*\\.*\.lock]
Sorry to dig this one back up but another that I suggest adding to the list for Windows related backups is \System Volume Information\ or set the option for --exclude-files-attributes to Hidden, System, Temporary. I tend to want to backup the Read Only files anyways, but this should cover that list as well.
Please be careful with this exclusion list and use it as a reference and not blindly copy/paste it.
I made that mistake. Not only was it excluding stuff I did want to backup, but there are some duplicate filters. Also some of those filters might simply do not apply to you(if you don’t have any apple software on your computer).
Another point is that I observed these exclude filters slow down backup time. After I pruned that list my backup time dropped from 55min to 26min on ~60k files and ~6-7GB of data.
Edit: Also looking at those regex’s, many of them will never match at all because Windows uses \ slashes, not /.
