FileAccessError while processing files

I’m seeing a lot of FileAccessError errors with Duplicati 2.0.4.30 when running backup jobs. There are hundreds to thousands of them depending on the size of the job. They are not always consistent with regards to the files that trigger the issue, however the error itself is consistent. The error as reported in the verbose file log is:

2019-10-16 15:44:29 -07 - [Warning-Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess-FileAccessError]: Error reported while accessing file: /mnt/SomeShare/SomePath/redacted/
System.IO.IOException: Interrupted system call
  at System.IO.Enumeration.FileSystemEnumerator`1[TResult].FindNextEntry (System.Byte* entryBufferPtr, System.Int32 bufferLength) [0x0004c] in <d59a6ed4e2f34efbabec0525e05bcea7>:0
  at System.IO.Enumeration.FileSystemEnumerator`1[TResult].MoveNext () [0x00054] in <d59a6ed4e2f34efbabec0525e05bcea7>:0
  at System.Collections.Generic.LargeArrayBuilder`1[T].AddRange (System.Collections.Generic.IEnumerable`1[T] items) [0x0003d] in <d59a6ed4e2f34efbabec0525e05bcea7>:0
  at System.Collections.Generic.EnumerableHelpers.ToArray[T] (System.Collections.Generic.IEnumerable`1[T] source) [0x00033] in <d59a6ed4e2f34efbabec0525e05bcea7>:0
  at System.IO.MonoLinqHelper.ToArray[T] (System.Collections.Generic.IEnumerable`1[T] source) [0x00000] in <d59a6ed4e2f34efbabec0525e05bcea7>:0
  at System.IO.Directory.GetFiles (System.String path, System.String searchPattern, System.IO.EnumerationOptions enumerationOptions) [0x00009] in <d59a6ed4e2f34efbabec0525e05bcea7>:0
  at System.IO.Directory.GetFiles (System.String path) [0x0000b] in <d59a6ed4e2f34efbabec0525e05bcea7>:0
  at Duplicati.Library.Snapshots.SnapshotBase.ListFiles (System.String localFolderPath) [0x00000] in <c26289f99970440d9746601b6632143c>:0
  at Duplicati.Library.Utility.Utility+<EnumerateFileSystemEntries>d__20.MoveNext () [0x00213] in <c1e0f814ef8949ae801eb07cd4d43d3d>:0

Duplicati is running on an Archlinux system with mono 6.0.0.327, the data source path is a samba share from a Synology NAS. Other files and directories from the same backup job from the same source are fine. This issue started within the last couple of releases (though I unfortunately can’t pinpoint it).

Has anybody else seen this issue? This is very concerning from a backup integrity standpoint.

Yes, it is… the most obvious cause is lack of permissions to the source data. Is the Duplicati process running under a UID/GID that has at least read access to all the files in your mount point?

For what it’s worth, it’s usually more efficient to back up data AT the source. Running Duplicati on your NAS (either native Synology package, or a docker container) may give you better results. Or maybe not if you have a really low-powered NAS. Just something to consider.

Thanks for the reply!

The permissions are definitely correct, and the files are largely processed correctly. When there are errors, it’s not all of the files, and the files that are affected are not consistent.

Running Duplicati at the source, on the NAS, won’t solve the issue – the issue would still be present, just potentially masked. It also removes the Duplicati VM itself from the rest of my lab infrastructure, making it an outlier for the rest of the automation processes that I have.

I’m wondering if this is OS specific, and if it’s perhaps a Mono version issue.

I don’t think that’s necessarily true since you wouldn’t be accessing the data through the Samba share. But I understand your other point and how you want to keep Duplicati where it is.

Have you tried looking at the Synology Log Center to view events related to file access? Curious if you see any problems there that might help shed light on this.

That’s a great suggestion to look at the Syno logs, however there’s nothing untoward there. I also don’t have any trouble accessing those files from the box that has the share mounted, as the duplicati user.

Does anybody else have any insight in to this issue? It’s persisting even after the latest Canary release.

There have been several reported issues with SMB/CIFS shares with Duplicati, as well as other applications. Can you try adding cache=none to the mount options for your share in your fstab? See the below for example:

It might take you to take a look. Interrupted system call is a specific thing that you could chase with strace (I think) to see if you can see what sort of interrupt is happening, and what system call is suffering. I think there are few system calls that aren’t done by mono, so this might be a mono bug…

You could also possibly confirm that the issue is purely in FileEnumerationProcess by trying to avoid actual backup in various ways. An easy one is to do a test backup on a small file set without change. Similar testing on non-Samba area would confirm whether or not the problem only occurs on Samba.

@warwickmm

Thanks for the link. I gave that a shot, to no avail – the issue persists.

@ts678

I may have to delve in to strace’ing this, as you mentioned. I have run the backup on a set with no changes, and the problem still occurred. It’s definitely occurring in FileEnumerationProcess. I may also need to duplicate the data to a non-samba location and try that as well.