Duplicati can't access file even after access rights have been granted


#1

One of my backups included a file (default.m3lib) to which it had no access rights. I changed the access rights using

NAS:~# chmod o+r default.m3lib 

so now it looks like this:

-rw----r-- 1 root   root     9537289 Aug  3 01:38 default.m3lib

I have verified that I can access this file (and duplicati is running as a service under my username) and it works fine.

But duplicati still keeps complaining about not being able to access it. Why?

6 Dec 2017 23:50: Message
Failed to process path: \\NAS\together\default.m3lib
System.UnauthorizedAccessException: Access to the path '\\NAS\together\default.m3lib' is denied.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   at Duplicati.Library.Snapshots.SystemIOWindows.FileOpenRead(String path)
   at Duplicati.Library.Main.Operation.BackupHandler.HandleFilesystemEntry(ISnapshotService snapshot, BackendManager backend, String path, FileAttributes attributes)

How does Duplicati access network shares?
How does Duplicati access network shares?
#2

I’m not strong with the Linux, but if you (the user under which Duplicati is running) belong to a group that as NOT be granted access to something will that “no group access defined” limitation override the read access you provided to Other?

Actually, when you say you “have verified I can access this file” that means you actually opened it up in something (like nano) right?


I’m not trying to hijack your topic but I recently had a similarly odd issue on a Mac where a backup to a USB drive created under the Tray Icon was inaccessible to Duplicati once it was shifted to the service - even if I gave everybody full rights to the drive and each path level down to the backup folder.

I never did figure out the issue - I just renamed the backup folder, did a Test Connection to create a new backup folder, then moved the archive files into the new folder and everything started working just fine. It was odd.


#3

Yes, very odd indeed. I hope we can figure this out. Looks like a bug, @kenkendk

Yes and no. I opened it on my windows machine, where the duplicati is running. I thought that is what counts: if I am able to open the file under my username and duplicati is running under my username, it should be able to open it too. In fact, since the access rights I granted are not user specific, virtually anyone should be able to access that file.

I suspect that duplicati might be asking for more rights than it actually needs…


#4

Ah. So this is a Windows Duplicati run accessing a mapped or UNC NAS share?

If that’s the case then I would think the permissions for the user connecting to the share are what matters.

If that’s not the case, why are you using chmod on a Windows box (I would assume via the Linux subsystem)?


#5

Yes, sorry, that wasn’t quite clear.

Well, yes. But apparently they don’t. That’s my point. The user (in fact: all users: -rw----r--) has permission to read the file but duplicati somehow doesn’t seem to realize that.


#6

There’s nothing for Duplicati to realize - it has exactly what access the share is allowing.

If this is the only file causing the error then I agree there’s something goofy going on, but if Duplicati can get to even one file on the share then we know it’s not a share level permissions issue.

Is it possible somebody / something else on or using the NAS has the file open and it’s really a “file in use” issue? It’s possible the share can’t adequately define that as a situation so it just tells Duplicati “no, you can’t have the contents of this file but I’m not going to tell you why”…


#7

What version of Mono are you on? I just ran into a few weird file permission problems that seem to have gone away with an upgrade to 5.4.1.6-9 (I was on 3.1).


#8

No, I don’t see how this is possible, since I can do any read operation I want with the file via windows explorer: I can open it and look at the contents, and most importantly, I can copy it to another folder.

I’m not using mono. I’m running duplicati on windows.


#9

How do you have Duplicati accessing the NAS content - is it one of these?

  • mount point (mklink or symlink) as a subfolder
  • mapped drive letter
  • UNC path
  • Swish (or something similar)
  • voodoo magic

#10

At this point, I’m actually pretty sure it’s voodoo.

But seriously: this is actually a good point. Because duplicati is accessing it via UNC path:

While when I access it manually, it is via mapped drive. (The reason why duplicati is using the UNC path is because it just won’t see the mapped drive)

However, I repeated the tests by accessing the file manually via the UNC path and it works just fine.


#11

The fact that you don’t see the mapped drive in the Duplicati file picker, seems to be an indication that the Duplicati server/service is running in a different user context than the account you logged on with.
This could explain why you can access the file interactively in contrast to Duplicati.
Are you 100% sure that you granted read permissions to the account that is used to start the Duplicati server/service?


#12

I agree with kees-z, but only if you can’t back up ANY files over this UNC connection.

If other things on the share are getting backed up EXCEPT this default.m3lib file then I thing something else is going on.

Just out of curiosity, are you running MusicIP on your NAS and, if so, does stopping that service allow the backup to run as expected?


#13

Yes. The service is running under the same user name as I am logged in with, i.e. my windows username:

That’s the thing. The backup job has successfully backed up about 600 GB:

image

The absolutely only file that fails is that one.

Yes, I’m running MusicIP on my NAS (well spotted!) but I don’t know for sure what happens when I stop the service, but I thought it just doesn’t make sense that it would change anything, since the file is obviously not locked. The thing is, I only fiddle with the MusicIP settings every other year or so and every time I have to look things up again, like how to start or stop the service etc. So I can’t check it right away…


#14

That’s fine. At the moment I only have a theory that MusicIP has the file open so the share isn’t allowing Duplicati access to it.

I realize this doesn’t match up with you being able to open the file manually, but it’s possible Duplicati is trying to exclusively open the file (so nothing else changes it while Duplicati is processing it) while the manual test open is fine with sharing access to the file.

On a local Windows drive the shadow copy service (if enabled) would handle sharing access while ensuring Duplicati’s view of it doesn’t change while open, but that doesn’t generally extend to network connections.


#15

If it’s only the file default.m3lib you’re having problems with, I guess @JonMikelV is on the right track.
In case it still turns out to be a permission problem (I don’t expect that), check if the location on your NAS is stored in the Windows Credential Manager.
The current version of Duplicati doesn’t make use of the Credential Manager, but there are plans to use the CM for storing passwords in future versions. So if credentials for accessing your NAS are stored in the CM, Duplicati isn’t aware of it.


How does Duplicati access network shares?
#16

Aaah, I didn’t know that there are different kinds of file access (exclusive and non-exclusive). So file access is not just a matter of granting access but also about requesting (the right kind of) access. :exploding_head:

I’ll check it.


#17

Yes, files that are opened exclusively cannot be read by other processes until the file is closed.

This is where VSS and LVM come in. These technologies create a consistent snapshot of all files, including open files. This snapshot is then used as backup source, making it possible to backup open files.

Snapshots do not work on remote shared folders, so you have to terminate the process that keeps the file open, or exclude the file from your backup (which seems to be the best approach for your situation).


#18

Yes, I am aware of this. But @JonMikelV added something else to that picture: a process can request either exclusive access (and be rejected because anothee process already has (non-) exclusive access) or it can request non-exclusive access (and receive it even though another process already has (non-) exclusive access).


#19

As far as I know, Duplicati does not open files exclusively. Read-Only access is the only requirement, which is not enough to open the file exclusively.
Snapshot technologies (VSS, LVM) are used to be sure that files don’t change while they are backed up.
I guess it’s not Duplicati, but MusicIP that opens the file exclusively.


#20

No, becauss then I wouldn’t be able to open it either…