Duplicati doesn’t seem to accept negative lookaheads in regex. I currently have two cases where I’d like to exclude all but a single file/folder, and was wondering how I could go about this using just the filters?
Exclude all of AppData\Local\Google, other than the \Chrome folder (attempt: -[.*AppData\\Local\\Google(?!\\Chrome)\\])
Exclude all of Plex Media Server, other than \Plug-in Support\Databases\com.plexapp.plugins.library.db (attempt: -[.*Plex Media Server\\(?!Plug-in Support\\Databases\\com\.plexapp\.plugins\.library\.db$)])
According to regex101 this should work, but I’ve had no luck in either doing it this way, or with using an include filter before or after. Am I doing something wrong here?
Edit: I ran the command line with test-filter on --exclude=[.*\\Plex Media Server\\(?!Plug.*)], it came up with this message, so appears to be not checking inside the folder at all.
Excluding path due to filter: E:\LocalBackup\TestDir\Plex Media Server\ => ([.*\\Plex Media Server\\(?!Plug.*)])
I’m not a Filters expert, but I think excluding a folder also excludes everything below.
Keep in mind that negative lookaheads are zero-width, so the Server folder matches.
Similarly in the first example, AppData\Local\Google\ matches. Maybe you can try
.*AppData\\Local\\Google\\(?!Chrome\\).+\\
I haven’t tested this, but that’s the idea. This excludes subfolders other than Chrome.
I’m not sure it excludes files right in Google folder. If you want to, maybe drop last \\.
Instead of trying to mess with exclude, could you just GUI-check the ones you want?
Thanks for the help, what you suggested mostly works, but only for a single subdirectory.
Works:.*AppData\\Local\\Google\\(?!Chrome\\).+
Works:.*Plex Media Server\\(?!Plug-in Support\\).+
Doesn’t work:.*Plex Media Server\\(?!Plug-in Support\\Databases\\).+
Wanted:.*Plex Media Server\\(?!Plug-in Support\\Databases\\com\.plexapp\.plugins\.library\.db$).+
Is there an easy tweak to get it working with more than one directory?.
I found by using a single regex filter, my backup speed went from 2h30 to 15m. I made a Python library to construct the filter, and I’m slowly trying to get everything added, so you can select an entire drive without having to worry about exclusions
Could you clarify the results please? When I see an RE that “Works”, I assume it works.
“Doesn’t work” means something wrong, but what exactly is the problem you’re seeing?
“Wanted” is not something I’d think of as an RE. It sounds like a path wrongly omitted…
Duplicati.CommandLine.exe test-filter E:/LocalBackup/TestDir --exclude="[.*Plex Media Server\\(?!Plug-in Support\\).+]"
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\
Excluding path due to filter: E:\LocalBackup\TestDir\Plex Media Server\Codecs\ => ([.*Plex Media Server\\(?!(Plug-in Support\\)).+])
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.dlna.db
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.dlna.db-shm
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.dlna.db-wal
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.itunes.db
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.itunes.db-shm
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.itunes.db-wal
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.library.blobs.db
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.library.blobs.db-2020-04-05
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.library.blobs.db-2020-04-08
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.library.blobs.db-2020-04-11
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.library.blobs.db-shm
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.library.blobs.db-wal
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.library.db
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.library.db-2020-04-05
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.library.db-2020-04-08
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.library.db-2020-04-11
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.library.db-shm
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\Databases\com.plexapp.plugins.library.db-wal
Matched 18 files (75.10 MB)
Not working once folder is added:
Duplicati.CommandLine.exe test-filter E:/LocalBackup/TestDir --exclude="[.*Plex Media Server\\(?!Plug-in Support\\Databases\\).+]"
Including path as no filters matched: E:\LocalBackup\TestDir\Plex Media Server\
Excluding path due to filter: E:\LocalBackup\TestDir\Plex Media Server\Codecs\ => ([.*Plex Media Server\\(?!(Plug-in Support\\Databases\\)).+])
Excluding path due to filter: E:\LocalBackup\TestDir\Plex Media Server\Plug-in Support\ => ([.*Plex Media Server\\(?!(Plug-in Support\\Databases\\)).+])
Matched 0 files (0 bytes)
If I had to guess, it looks like it needs an exclusion on each part of the directory so it’ll continue to look inside, but I don’t know regex well enough to figure out how to pull that off.
Just to clarify, I will likely be using similar regex in other cases, and I’m also wanting to learn a bit more regex while doing this, since I’ve spent years with Python and completely ignored it
Likely the same problem where excluding a parent excludes its subfolders. Fix (although taking this path feels like a tangle of double-negatives) might be to make sure needed folders aren’t excluded. Example:
Duplicati.CommandLine.exe test-filter “C:\tmp” --exclude="[C:\tmp\Plex Media Server\(?!$|Plug-in Support$|Plug-in Support\Databases$|Plug-in Support\Databases\com.plexapp.plugins.library.db$).+]
I’m using $ anchors inside the alternation for clarity. Same reason I’m not simplifying path match with ?
It can be complex. O’Reilly has lots of books. Other publishers might too. Consider using comments
Thanks for the help again. That one in particular didn’t work for me, but I used your idea and found an alternative that did, each folder requires another level of recursion if you’d call it that. To be honest it does seem quite hacky, so I’ll leave this for the rare cases haha.
.*Plex Media Server\\((?!Plug-in Support\\)|Plug-in Support\\((?!Databases\\)|Databases\\(?!com\.plexapp\.plugins\.library\.db$))).+
I’ve gotta admit, I’m in sort of a love-hate relationship with regex lol. For years in Python I absolutely refused to use it, but I’m slowly warming to just how useful it actually is. I’m happy just learning the basics right now, the stuff in the link looks a bit hard to say the least
