Duplicati - 2.0.7.1_beta_2023-05-25
I found this repo:
But it doesn’t work, the filter looks really weird when imported, I think it’s made for an older version of Duplicati.
I’m not saying these are good filters, but it’s a start, there are definetely some of them I wouldn’t like to use.
So anybody that has some good filters for windows?
Welcome to the forum @fribse
Weird how? Imported how? It’s probably meant to be put in Filters three-dot menu Edit as text.
Filters in User Manual also mentions the built-in filter groups. They’re a bit old, but have you looked?
Because they’re user-sensitive, a good way to get a personalized view is GUI Commandline and run
help, with no Target URL in that box, and with filter-groups in the Commandline arguments box.
Yes, I do import them as text:
Your output is pretty strange. For the 63 input lines, I get 63 entries in Edit as list, basically one for one. Your first entry with a label is the Videos one that should be at line 57, but you got it as entry 3…
In case there’s some weirdness with invisible characters, such as line endings, maybe just type a few.
are my first 4 entries which look about right for the first 4 lines what we both are showing for raw input.
Indtast udtryk her seems to be Danish for Enter expression here, so did that pick up nothing?
Again, trying a bit of it at a time, manually, might be worth doing to see if the resulting view gets better.
EDIT 1:
In Danish, it looks like:
EDIT 2:
2.0.7.100_canary_2023-12-27 did have some fixes, but I’m not sure if they explain all symptoms here.
Fix various bugs with backup filters, thanks @Jojo-1000
EDIT 3:
2.0.7.1 Beta looks pretty good to me. I’m assuming you’re on at least that. If not, maybe upgrade to it?
it may be the case, as with recent Duplicati ‘exclude file’ with ‘.rbf’ gives this as text:
-*.rbf*\
and not
-.rbfSome of the filter builder outputs have always looked a little suspect (not the way I’d hand-code them). What’s going on here is in the opposite direction (always a bit of guesswork), and it seems way wrong.
That’s why I’m suggesting testing a little at a time to see if the reversal can do better than being empty.
no scratch that I don’t understand how I could be so mistaken, I tried it again and it’s not doing that.
Edit:
-.rbf\
translates to ‘exclude directory’ with ‘.rbf’ as value. But
-.rbf
does not translate to anything.
The filter list you linked seems to consist of incorrect filters (or maybe leading * were removed).
There are some pre-defined sets of filters (“Filter groups”) that are somewhat hidden:
Note that “System Files” here means something different than under Exclude. It is path based instead of file attribute based. These filters are adapted to the OS, but I don’t think they were specifically updated for windows 11 (the only distinction is mac/win/linux).
Since these filters are not very well described, here is the internal list for my system:
SystemFiles:
-*\I386*
-*\Internet Explorer\
-*\Microsoft*\RecoveryStore*
-*\Microsoft*\Windows\*.edb
-*\Microsoft*\Windows\*.log
-*\Microsoft*\Windows\Cookies*
-*\MSOCache*
-*\NTUSER*
-*\UsrClass.dat
-?:\$Recycle.Bin\
-?:\hiberfil.sys
-?:\pagefile.sys
-?:\Recycled\
-?:\Recycler\
-?:\swapfile.sys
-?:\System Volume Information\
-C:\ProgramData\Microsoft\Network\Downloader\*
-C:\ProgramData\Microsoft\Windows\WER\*
-C:\Users\User\AppData\Local\Temp\\*
-C:\Users\User\index.dat
-C:\Windows\Installer\
-C:\Windows\memory.dmp
-C:\Windows\Minidump\*
-C:\Windows\netlogon.chg
-C:\Windows\softwaredistribution\*.*
-C:\Windows\system32\LogFiles\WMI\RtBackup\*.*
-C:\Windows\system32\MSDtc\MSDTC.LOG
-C:\Windows\system32\MSDtc\trace\dtctrace.log
OperatingSystem:
-*\Recent\
-?:\autoexec.bat
-?:\Config.Msi*
-C:\Users\User\AppData\Roaming\Microsoft\Windows\Recent\
-C:\Windows.old\
-C:\Windows\
CacheFiles:
-*\AppData\Local\AMD\DxCache\
-*\AppData\Local\Apple Computer\Mobile Sync\
-*\AppData\Local\Comms\UnistoreDB\
-*\AppData\Local\ElevatedDiagnostics\
-*\AppData\Local\Microsoft\Edge\User Data\Default\Cache
-*\AppData\Local\Microsoft\VSCommon\*SQM*
-*\AppData\Local\Microsoft\Windows Store\
-*\AppData\Local\Microsoft\Windows\Explorer\
-*\AppData\Local\Microsoft\Windows\INetCache\
-*\AppData\Local\Microsoft\Windows\UPPS\
-*\AppData\Local\Microsoft\Windows\WebCache*
-*\AppData\Local\Packages\
-*\Application Data\Apple Computer\Mobile Sync\
-*\Application Data\Application Data*
-*\cookies.sqlite-*
-*\Dropbox\Dropbox.exe.log
-*\Dropbox\QuitReports\
-*\Duplicati\control_dir_v2\
-*\Google\Chrome\*cache*
-*\Google\Chrome\*Current*
-*\Google\Chrome\*LOCK*
-*\Google\Chrome\Safe Browsing*
-*\Google\Chrome\User Data\Default\Cache
-*\Google\Chrome\User Data\Default\Cookies
-*\Google\Chrome\User Data\Default\Cookies-journal
-*\iPhoto Library\iPod Photo Cache\
-*\Local Settings\History\
-*\Mozilla\Firefox\*cache*
-*\OneDrive\.849C9593-D756-4E56-8D6E-42412F2A707B
-*\permissions.sqlite-*
-*\Safari\Library\Caches\
-*\Temporary Internet Files\
-*\Thumbs.db
-C:\Users\User\AppData\Local\Microsoft\Windows\History\
-C:\Users\User\AppData\Local\Microsoft\Windows\INetCache\
-C:\Users\User\AppData\Local\Microsoft\Windows\INetCookies\
TemporaryFiles:
-*.tmp
-*.tmp\
-*\AppData\Local\Temp*
-*\AppData\Temp*
-*\Local Settings\Temp*
-C:\Users\User\AppData\Local\Temp\
-C:\Windows\Temp\
Applications:
-C:\Program Files (x86)\
-C:\Program Files\
-C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Administrative Tools\
-C:\Users\User\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Administrative Tools\
DefaultExcludes:
SystemFiles | OperatingSystem | CacheFiles | TemporaryFiles | Applications
What you are looking for is probably Default excludes (everything combined), but check that there is nothing else missing from that list, as it is not very up to date.
Right now, it is not displayed in the UI which files will be excluded by these filters, but I made an update for the future. You can also just look at that list to see what you need and add it manually, then it should be shown correctly.
Hi @ts678
Thankyou for your replies, they at least highlight a big problem for me, I’m not the only one that doesn’t understand the filters 
And thankyou for showing the GUI in danish, it’s not necessary, but it’s a very kind thought! 
I’m on the beta (which is the fist line in the original post), and I’m not sure canary is a good idea for me
I will try and build the filters, and see what happens, thankyou again!
Thankyou for the list, that is very interesting.
Even more interesting is it, that if I copy your list, it shows the same weird formatting as the gist I found originally.
What version of Duplicati are you using?
If I try and add some of your filters manually:
Like this:
-*\I386*\
-*\Internet Explorer\\
-*\Microsoft*\RecoveryStore*\
-*\Microsoft*\Windows\*.edb\
-*\Microsoft*\Windows\*.log\
-*\Microsoft*\Windows\Cookies*\
-*\MSOCache*\
Hmm, there is really something wrong with the filter interpreter.
Entering
“Exclude file extension” → tmp or .tmp or *.tmp
Just blanks out the ‘method’
“Exclude files whose names contains” → *.tmp is accepted, but I have no idea if that works yet.
I’ve also tried entering some of the filters by hand (buest guess in method):
-*\I386*\
-[.*\Internet Explorer\[^\\]*]
-*\Microsoft*\RecoveryStore*\
-[.*\Microsoft*\Windows\*.edb[^\\]*]
-[.*\Microsoft*\Windows\*.log[^\\]*]
-*\Microsoft*\Windows\Cookies*\
-*\MSOCache*\
-[.*\Recent[^\\]*]
-C:\Users\User\AppData\Roaming\Microsoft\Windows\Recent\
-C:\Windows.old\
-C:\Windows\
-[.*\Internet Explorer[^\\]*]
-?:\$Recycle.Bin\
-?:\Recycled\
But if I run the backup job I get:
(probaby more errors, but this is the first)
Below original post is a link to another topic on the weird look. It looks like Firefox 123.0 does it.
Is that what you’re using now? Did it used to look better, even on the immediately prior Firefox?
I tried this very syntax with Duplicati 2.0.7.1 beta on a Windows server and I got:
analyse de ".*\Internet Explorer\[^\\]*" - Séquence d'échappement \I non reconnue
2 interesting nuggets of information:
I don’t have much time to investigate this further just now.
Edit: just add this:
System.ArgumentException: analyse de ".*\Internet Explorer\[^\\]*" - Séquence d'échappement \I non reconnue.
à System.Text.RegularExpressions.RegexParser.ScanCharEscape()
à System.Text.RegularExpressions.RegexParser.ScanBasicBackslash()
à System.Text.RegularExpressions.RegexParser.ScanRegex()
à System.Text.RegularExpressions.RegexParser.Parse(String re, RegexOptions op)
à System.Text.RegularExpressions.Regex..ctor(String pattern, RegexOptions options, TimeSpan matchTimeout, Boolean useCache)
à System.Text.RegularExpressions.Regex..ctor(String pattern, RegexOptions options)
à Duplicati.Library.Utility.FilterExpression.FilterEntry..ctor(String filter)
is unclear about the Firefox testing level, but 2.0.6.3_beta_2021-06-17 is mostly blank for me. 2.0.7.100_canary_2023-12-27 is too, so lack of displays seems to be related to the latest Firefox.
tried with Firefox ESR (115), no problem with it.
-[.*\Internet Explorer\[^\\]*]
which seems strange to me, as Character Escapes in .NET seems to say \I is an I not an error.
regexstorm.net claims to aim specifically at .NET regex (they do vary a bit) and shows the oddity.
This problem probably arises in the user’s best guess of usage not matching filter builder’s guess.
Filter builder was probably not expecting backslashes in Exclude files whose names contain because on Windows this can’t happen. There will be backslashes before, but none after a name.
That last point is probably what the right hand side of the regex is enforcing. No backslash to right.
Testing Exclude directories whose names contain and Internet Explorer looks plausible:
-*Internet Explorer*\
and seems pretty stable on different Duplicati versions and different browsers, so a different issue?
The problem with “exclude file whose name contains” is, it is only supposed to match the filename part. So it shouldn’t contain slashes in the first place, and internally it uses a regexp without escaping.
I was careful to mention Windows when making that point, because a backslash is legal on Linux:
$ cd /tmp
$ cat /dev/null > 'back\slash'
$ ls -ln back*
-rw-rw-r-- 1 1000 1000 0 Feb 26 13:34 'back\slash'
$
So the filter was sort of doomed from the start, but I still don’t understand why \I isn’t treated as I.
I tried entering the gist filter set with Edge, and that worked nicely!
Now I’ve gone from 700+ warnings to 47 warnings, and I don’t understand why I still get those.
The warnings still shown:
C:\Users\glbfr\OneDrive\Kom godt i gang med OneDrive.pdf
C:\Users\fribs\AppData\Roaming\Thunderbird\Profiles\67pokubp.default-release\parent.lock
C:\Users\fribs\AppData\Roaming\Mozilla\Firefox\Profiles\chq0x22w.default-release\parent.lock
C:\Users\fribs\AppData\Local\Mozilla\Firefox\SkeletonUILock-c388d246
C:\Users\fribs\AppData\Local\Microsoft\Edge\User Data\lockfile
C:\Users\fribs\AppData\Local\Microsoft\Edge\User Data\Default\Cache\Cache_Data\data_0
C:\Users\fribs\AppData\Local\Microsoft\Edge\User Data\Default\Cache\Cache_Data\data_1
C:\Users\fribs\AppData\Local\Microsoft\Edge\User Data\Default\Cache\Cache_Data\data_2
C:\Users\fribs\AppData\Local\Microsoft\Edge\User Data\Default\Cache\Cache_Data\data_3
C:\Users\fribs\AppData\Local\Elgato\Stream Deck\net.voicemod.windowsdesktop\Local Storage\leveldb\LOCK
C:\Users\fribs\AppData\Local\Elgato\Stream Deck\de.perdoctus.streamdeck.homeassistant\Local Storage\leveldb\LOCK
C:\Users\fribs\AppData\Local\Elgato\Stream Deck\com.elgato.youtube\Local Storage\leveldb\LOCK
C:\Users\fribs\AppData\Local\Elgato\Stream Deck\com.elgato.volume-controller\Local Storage\leveldb\LOCK
C:\Users\fribs\AppData\Local\Elgato\Stream Deck\com.elgato.obsstudio\Local Storage\leveldb\LOCK
C:\Users\fribs\AppData\Local\Elgato\Stream Deck\com.elgato.controlcenter\Local Storage\leveldb\LOCK
C:\Users\fribs\AppData\Local\Elgato\Stream Deck\cache\QtWebEngine\net.voicemod.windowsdesktop\Cache\Cache_Data\data_0
C:\Users\fribs\AppData\Local\Elgato\Stream Deck\cache\QtWebEngine\net.voicemod.windowsdesktop\Cache\Cache_Data\data_1
C:\Users\fribs\AppData\Local\Elgato\Stream Deck\cache\QtWebEngine\net.voicemod.windowsdesktop\Cache\Cache_Data\data_2
C:\Users\fribs\AppData\Local\Elgato\Stream Deck\cache\QtWebEngine\net.voicemod.windowsdesktop\Cache\Cache_Data\data_3
C:\Users\fribs\AppData\Local\Elgato\Stream Deck\cache\QtWebEngine\net.voicemod.windowsdesktop\Cache\Cache_Data\index
I made some filters that should hit them:
But it still shows warnings?
It’s probably me just not understanding the weird filter settings for duplicati, can somebody point me to my mistakes?