Excluding MS Office temporary files?


#1

I got a failure in my last backup job where Duplicati complained that it couldn’t access one of the temp files created by Office (in this case Excel) while a normal document is open. These files begin with “~$” followed by some version of the normal file’s filename. I created a filter
-~$*.*
attempting to exclude any such file, without having to exclude all hidden/system files (though I’d be willing to be convinced to hide system files anyway, i’m not sure what I feel on the matter…)

Anyway, I ran it again with the same file open and still got the same error message on that file. I’m pretty sure the syntax is correct at least for traditional windows filesearch conventions, but I guess it didn’t work for Duplicati - anyone have any advice?


#2

Try using this regular expression: -[.*\\\~$[^\\]*]


#3

The error message that gives when attempting to run:

parsing “.\\C:\Users\username$[^\]” - Unrecognized escape sequence \C.


#4

Yeah, that is a stupid bug that I really should fix:


Workaround for now is to use the ASCII code for the tilde, so the expression becomes:

-[.*\\\x7E$[^\\]*]

#5

Cool, that expression seems to be accepted. I still seem to get the “unable to access” error message though:

The process cannot access the file ‘C:\Work\Projects\~$MyExcelFile.xlsx’ because it is being used by another process. [edit: corrected the missing backslash here]

Also out of curiosity - it occurred to me that selecting to exclude “system files” might also work for this use case, but there’s not much explanation given as to what this will actually exclude (is it only the system-attribute windows files that are hidden by default unless you override them in explorer’s settings) - I’m nervous that checking this option will exclude some specific files that I’d really want to back up, and/or not exclude ones I’d expect to be excluded (such as above) - is there a place with any sort of in-depth explanation for this option?

Edit to add: I tried enabling the setting to exclude System files, and still got this warning - but I checked and the temp file is just hidden, not a system file as far as I can see, and I really don’t want to exclude hidden files from general backup.


#6

Yes, it is an attribute filter only.

Not really, you can see the description under the “Advanced options” area, and then choosing the option --exclude-files-attributes. The checkboxes are just a nicer interface for that option.

In that case you may want to go to the “commandline” section where you can run the “test-filters” command. But looking at the filename:

If is not a filename that starts with ~$, it is a filename that has ~$ in the middle. To match that, you need to use:

-[.*\x7E$[^\\]*]

It will match any filename that contains ~$ where the original matches files that start with ~$.


#7

My bad - I didn’t notice that the forum formatting removed a backslash out from under me :frowning:

The proper filename (needed to use escaping) is:

C:\Work\Projects\~$MyExcelFile.xlsx

run the “test-filters” command.

I’ll give this a shot.


#8

Test Filters (after pointing it exclusively to the directory with the problem file) still includes the ‘~$’ filename, even with your original regex exclusion in place.


#9

The output should explain which filters it applies and why they match or not.

Do you see anything that could explain it?


#10

Nothing - the file in question gets its own “Including file:…” line, and no mention in any of the “Excluding” lines. I even added a different filter to the test set to see what the functionality would do (excluded all files with a certain normal string in the filename).

I now notice it also works if I just filter on $ - but of course that would accidentally risk filtering out legit files that happen to have a “$” character in the filename.


#11

Arh, the formatting took away the backslash.

\x7E is the right sequence for ~, so it must be like tis for matching only the start of the filename:

-[.*\\\x7E$[^\\]*]

And this for matching anywhere:

-[.*\x7E$[^\\]*]

#12

I have edited the original posts so the sequence is correct for others seeing the topic.


#13

I still get the following result when trying either of those in test-filters:

Including file: C:\Work\Projects\~$myExcelFile.xlsx (165 bytes)

:cold_sweat:


#14

Seems that another backslash has escaped (in front of the $):

[.*\x7E\$[^\\]*]

This works for me:


#17

When I set it up like you (with the path in the Target URL and the exclude in the Commandline arguments box) i now get the same result. But the way I’ve been running the test up until now (since this is the only way I can figure out to get to the commandline easily) is by selecting “Commandline…” on an existing backup –
image

Here I get the Target URL pointing to my backup file location, the Commandline Arguments pointing to all the folders I’m backing up, and the “exlude” is one of the members of the “advanced options” list. When I have the “exclude” listed here, it seems to not work at all still (even though it appears exactly the same way in its command line argument). I just tried removing the other “exclude” statements from the list in “edit as text”, since i’m overriding the “commandline arguments” box to point only at the one folder contianing the file in question, and it works that way. I’m a little confused as to what’s happening to it otherwise.

Edit: i’ve updated the regex exclusion on the backup set with your newest version and it does appear to work (regardless of the confusion when doing test-filters). I never thought of escaping the ‘$’ character for some reason. Thanks for the help!


#18

BTW, is there any way to edit the files a backup job points to without needing to run all the way through the configuration wizard over and over? That gets pretty tedious, and is a good way to accidentally let errors creep in…


#19

That pre-fills the commandline screen, which is supposed to make it easier to run the exact command that the backup operation runs.

I could not find a good place to put in the “clean” commandline link, so there is none, but you can just remove the number at the end of the url to get a clean commandline page. The url would then look like:

http://localhost:8200/ngax/index.html#commandline

Not sure exactly what happens there, but perhaps there is some mix of options and arguments that messes things up.


#20

Hi, just jumping in the middle here.
I’m having the same issue here.
Where should I add the --exclude=[.*\x7E\$[^\\]*] expresion? Should I add it under “advanced options” (see attached)

Also, please clarify, does this expression simply escapes every file that has a “$” in it? will that escape folders as welll?

Thanks


#21

Hi @YMtech, welcome to the forum!

Filters are added on step 3 (Source Data) of editing a job, however it looks like you are trying to add it as an advanced parameter which is done on step 5 (Options). Either will work.


As a regular filter (step 3)

You’ll need to expand the “Filters” section, then either:

  • select the “Add filter” link, select the filter type from the list, then put your filter (WITHOUT the --exclude= prefix) in the field to the right

  • select the “Edit as text” from the “three dots” menu on the right, then put your filter in the text box that appears (WITHOUT the --exclude= prefix)

image


As an advanced parameter (step 5)

You’ll need to expand the “Advanced options” section, then:

  • click the “Edit as text” link from the “3 dots” menu on the right
  • put your whole exclude (including --exclude=) on it’s own line

#22

Thank you for your reply.

I’ve tried with the filters, it doesn’t seem to be excluding these files. I’m still ending up with the same error. It seems to revert back to “exclude file” instead of “exclude expression” even after I save the settings.

Any ideas?