Desktop.ini generates a warning log when I backup google drive folders

Either way including more may benefit in better diagnosing. Notepad++ opens the large file very quickly so it makes little difference in including more. Just shouldn’t be posted online :slight_smile:

Oh, so the “exclude file” can also be used with wildcard? They should probably make the UI more clear. Think I tried that before with something there to no effect.

Maybe I tried *filename instead of *\filename. To me, either should work.

Oh. Whoops. Too tired apparently. I looked before I posted and was sure it was the same at the time lol.

Yeah, it still contains an ex message for error that isn’t found here so it should work out the same anyway for diagnosing. Need that message.

Maybe, unless it doesn’t. Too much information makes relevant information harder to find.
Try to “skim” a Profiling log. For even more fun, add --profile-all-database-queries

  --profile-all-database-queries (Boolean): Activates logging of all database
    queries
    To improve performance of the backups, frequent database queries are not
    logged by default. Enable this option to log all database queries, and
    remember to set either --console-log-level=Profiling or
    --log-file-log-level=Profiling to report the additional log data
    * default value: false

It’s best to not assume users have specific software or to expect installation and learning.
Another that can open huge files is EditPad Lite. Specialty tools like glogg/klogg exist too.
My big logs look like they’re about 20 GB. At some size, you don’t want it all in memory…

I’m not sure. It would be most certain what you get if you type it into Exclude expression.
The GUI tries to help, but as you say it’s not clear exactly how to use it. Manual is weak too.
One way to see what the GUI builds is with Edit as text as mentioned. Exclude file is

-*\default.ini

so (if I tested right), it did what I wanted, and made the same filter as Exclude expression
however I can’t promise it would always do that. Much testing and JavaScript study needed.

“They” is the community. If nobody (including you) will help with UI or docs, change is slow.
Filters are a pretty messy area, easy to get wrong. Arguably, Exclude by attribute is worse.

Regardless, we seem in agreement that we need the message details, obtained somehow.
If you look at these much, does it look to you like logging two lines instead of one will help?
Maybe that could be a GitHub request, although these days a pull request will be far better.

Example:

2019-03-25 18:55:53 -04 - [Warning-Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess-FileAccessError]: Error reported while accessing file: C:\PortableApps\Notepad++Portable\App\Notepad++\updater\
Alphaleonis.Win32.Filesystem.DeviceNotReadyException: (21) The device is not ready: [\\?\GLOBALROOT\Device\HarddiskVolumeShadowCopy68\PortableApps\Notepad++Portable\App\Notepad++\updater\]
   at Alphaleonis.Win32.NativeError.ThrowException(UInt32 errorCode, String readPath, String writePath)
   at Alphaleonis.Win32.Filesystem.File.GetAttributesExCore[T](KernelTransaction transaction, String path, PathFormat pathFormat, Boolean returnErrorOnNotFound)
   at Duplicati.Library.Snapshots.SystemIOWindows.GetFileAttributes(String path)
   at Duplicati.Library.Utility.Utility.<EnumerateFileSystemEntries>d__23.MoveNext()

Logging the whole stack trace, while sometimes helpful (there’s that issue of “how much” again)
might be a bit much. Some can get long. The first line after the logged error often seems helpful.
I don’t know the logging code, but if it now grabs only one line, maybe it can pick up two instead.

I’m only commenting on one other thing here and I’m sure its included in the issue in some way.

If you do “exclude file” of eg “**.lock” (one star the formatting is being a pain) and (so “*.ini” should as well) then after hitting next or moving on to a different page and go back then it will have switched it to “file extension”.

That means they are doing changes in the code as to what the user chooses. Which I think is why I’m seeing excludes changing compared to what I originally did and am scratching my head.

When I originally excluded the folder in my first post in here, I was sure it was set right and if Duplicati incorrectly adjusts it then it wouldn’t have worked and I’d be left thinking it was a bug with Duplicati which is what I did.

Either way, the user is specifically entering in selections and values. It probably should not be adjusted that way. Validation to me should be to alert the user as its being entered, not to adjust blindly after the fact. The user should be aware or confusion is chaos.

Not quite. User input is turned into filter syntax. When GUI needs it again, that’s reversed.
This tries (not sure how well it succeeds) to explain what user did, using dropdown labels.
You can also look at Commandline or job export to view --exclude and --include lines.
Those match the minus and plus signs in the filter syntax seen in the GUI three dot menu.

I’d have to look at the code but its messed up. If the user chooses exclude file then it shouldn’t try to change it to file extension without the user deciding that or otherwise knowing about it. It just blatantly does it. That’s the main point.

While its correct in this case that doesn’t mean all cases are and I’m sure I’ve run into incorrect ones or I’m eating too many mushrooms.

So, if say you do “exclude folder” folderName then next and go back as you want to add a path then its changed to “exclude dir whose names contain” and you enter the path and now its borked. You selected “exclude folder” and have to re-look at it. That was my issue.

Now however, after you change it back to “exclude folder” and next and back and it will no longer change even without a path. So its better at that point.

I’m not sure what you’re typing in the reply but there’s a bug here lol :slight_smile:

This is why I wanted to focus on the topic and not get lost in a filter discussion. But here we are.

If I do Exclude file of *.ini (using backticks while typing that, to reduce forum reformatting):

-*.ini

is the filter expression in Edit as text

If I do Exclude extension of ini the filter builder builds the same thing. Two ways to get that.
Duplicati does not remember the way, so on the return trip it picks one that works – the second.

There’s been talk of trying to find a way to remember the GUI dropdown choice, but remember
that whatever is devised needs to fit within the existing option system like Commandline shows.
If you have a proposal at that level, feel free to suggest how it be done. Better yet, go and do it.

If you think the code is doing something wrong, that’s why you’d look. I think some code is here.
The goal is either to do an arguably-correct (which is hard to do with the unclear rules around it)
round-trip from the stored options, or find some tolerable way to remember the actual dropdown.

Was making a reply to an earlier version of your post, which you have edited three times – so far.
Let me see if I can figure out the new addition which gives an actual case (though not very clear).

Exclude folder of folderName makes -*folderName*\ which seems a bit looser than asked.
Using Edit as list calls that Exclude directories whose names contain of folderName.
This seems a correct interpretation of the seemingly extra-loose filter string that the GUI created.
Using Edit as text to remove the extra-loose trailing * (before backslash indicating folder) got
Exclude folder of *folderName. If adding path is important to your case, please describe well.
“borked” is not an adequate description. Step by step, with details please (or maybe my test is it).

1 Like

My edited reply has the steps to reproduce. This is with current beta. I’d be surprised if its been fixed in master as of yet.

I feel this is still on topic and its a possibility as to what happened. There’s no way to know if any user with an incorrect filter type ran into this.

But basically:

  1. add new filter

  2. change value to “exclude folder”

  3. enter folder name (any name will do)

  4. next

  5. previous

  6. Notice filter type has changed.

  7. Change back to “exclude folder”

  8. Next

  9. Previous.

  10. Notice its now correctly staying as “exclude folder”

Note: Path doesn’t matter. You don’t need to enter one. Its just changing the type during one browse back while not changing it during the other after second type change. So its having two minds.

I had to edit a bunch of times as its all happening live lol.

I’m trying to find the code for this. Any idea where it is? So far my searches won’t find the specific class.

I’m just about done with it though. Now that I know about it, I don’t feel so crazy. If they fix it or not its up to them. I probably won’t other than make a git issue at most. End of edits lol.

Using a wrong filter (with help from little documentation) probably allowed files access.
Beyond that, I don’t see how a filter itself could make an access warning, but we’ll see.

A test could be done without any filters at all, e.g. backing up one specific troubled file.
The easy test (if Duplicati is running as this user, but we don’t know) may be notepad.
For that matter, type would probably do. The default.ini I found are just text inside.
Why the ones in the OP can’t be accessed is unclear. That’s the primary question IMO.

There is no such thing as a filter type internally in C#. That’s an artificial concept created in GUI.

I explained all of this just above. Two ways to get to filter expression. Reverse trip must pick one.

Yes, and I gave you the link:

EDIT:

and I suspect rx helps decide what it is, and splitFilterIntoTypeAndBody later may be part of that.
My JavaScript is not enough to pursue this, but you might be amused that I’m searching in Notepad++.

You seem to be misunderstanding me. Eg when I say “filter type”, I’m not referring to c# .

I can reproduce two effects on Duplicati most recent release from git here using a fresh OS that hasn’t had Duplicati before and doing the provided steps. Duplicati does two different things when it should be doing one.

You also misunderstand that the user above has a wrongly inputted value to the exclude file so that it will not exclude all desktop.ini files. The issue I present clearly can cause this to happen and is 100% relevant as it will change the original choice. Oh, the error itself is besides the point if the exclude had worked it wouldn’t be a thing here.

I will deal with it on my own. Thanks anyway :slight_smile:

@rgheno give this a try, from my quick testing it seems to work.

Use “Exclude regular expression” with \w\:\\.*\\desktop.ini|\w\:\\desktop.ini

Tested against these folders at regexr.com

C:\test\desktop.ini
D:\abc\def\desktop.ini
F:\desktop.ini
C:\users\my user\desktop\new folder\important stuff\desktop.ini

After adding the filter to a test backup job. Now in the GUI when I select any folder that contains a desktop.ini file, it now excludes that file with a red X all on it’s own.
You may be able get away with just using \w\:\\.*\\desktop.ini, the |\w\:\\desktop.ini is only for files directly off the root like “F:\desktop.ini” otherwise if it’s in a subfolder the first part should be enough.

Hope that helps.

You refer to GUI, right? C# reference was to say (correctly I hope) that GUI filter type is a step removed.
I think ultimately what matters in terms of the C# code and actual filtering is what Edit as text shows.

No misunderstanding, but a politely stated request that we chase the access warning instead. Notice:

Furthermore:

This is where we diverge. That’s the mystery that I want to solve. Fixing filters “should” be easy.
We both suggested versions that should do the job, but got caught in JavaScript GUI oddities…

I’d note that the original post has an Exclude file of desktop.ini, and that one seems solid.
At least it passes the Next → Previous test, but it was the wrong syntax for the desired result…

I think I see what you’re getting at. You’d like step 10 to show the same as step 6, correct?
Both follow the visual pattern of Exclude folder and a folder name, yet the result varies?
Did you look at Edit as text to see what was set up? That explains this, but worries me.

Step 6 has left actual filter at -*someFolder*\ which is the extra-loose one I’d mentioned.
Step 10 has left actual filter at -someFolder\ which is quite a bit different in wildcard uses.
That might be why it sticks, but it’s probably not what you actually mean to be filtering with.
This gets back to the issue of there not being a clear definition (that I find) of GUI behavior.

I’m still not sure what things you mean, but I see identical GUI view making different filters.
On the return trip back to list view after Previous, dropdown varies, likely from the above.

Interesting. I wonder how smart those indicators are? I can’t get non-regex wildcard to redden things.
I actually had to do backup with *\desktop.ini wildcard to test, but a restore did in fact lack the file.

As a syntax nit, the period in desktop.ini for a regular expression technically deserves backslash too.
Do you know of any advantage in this case other than wildcard apparently doesn’t do color change?

Regardless, there’s yet another option to filter out the troublesome files, but I’d still like to know what FileAccessError is being raised. All of this depends on what @rgheno is willing to do to help look.

You are correct, the proper syntax should contain desktop\.ini but doesn’t seem to change the result either way for some reason… (I’ll look into it further tomorrow). The auto marking of excluded files was a completely unexpected bonus.

Advantages, Duplicati seems to like working with regex filters and so long as your expression is correct you get the expected results. I know most people probably don’t want to use regex but given it’s an option, I figured I’d mention it.

Here is a slightly smaller expression \w\:\\.*\\?desktop\.ini, that catches any “desktop.ini” file regardless of path depth.

FYI: You can change the filename to any filename you want. To exclude a file named “mySecretWords.txt” change the expression to \w\:\\.*\\?mySecretWords\.txt.

Thanks for helping me.
However, this expression just added 1 warning (it was 208 and now it’s 209).

I’ve also tried with just \w:\.*\desktop.ini and the result was the same.

I got lost in the discussion here haha that’s why I haven’t replied to the posts, I’ve just let it cool down a little bit.

I’m definitely willing to help, specially if this issue is common and will help the community.
Just let me know what you need, screenshots, logs, etc…

This expression with the “?” also gave me and extra warning.

image

to understand what is the issue with these files, what is needed is logdata. Unfortunately I don’t know how to show logdata with the Web UI reporting, however this logdata is included in mail reports and you will get the detailed reason why your desktop.ini files are not backed up. If the messages are not clear for you, post one example here (no need to include 208 messages, thanks in advance).

Agreed.

Viewing the Duplicati Server Logs is what you’re doing, and it’s GUI-only because only GUI has server.

If you wind up increasing from Warning to Verbose, you might want to log to a file, as it might get large.

log-file=<path> and log-file-log-level=Verbose or whatever winds up being appropriate to dive deeper…

My suggestion is above. Logs (click on lines too), then either screenshot or copy-and-paste into forum.

I can’t edit my previous post so I’ll just reply back to it.

A quick note about using \w\:\\.*\\?desktop\.ini or my other suggested filter expressions, they only work in Windows.

If you’re using a Mac or Linux you will need to adjust for the mount point vs \w\:\\ and change the paths to use of / vs \

I was re-reading the Duplicati filters page for another reason and I figured I should mention it.