Missing files from backup (drive mounted as folder)

Hello,

System specs: Windows server 2008 running Duplicati 2.0.1.2
Duplicati is running as a service, the system has been rebooted after setting it up.

I have a RAID-1 drive which is mounted as a folder in C:\Users\Administrator\Downloads\

After backing up the folder is there but there are no files in the folder, the folder is empty.

I wanted to set up a backup for that folder to see what happens, but when I checked the folder it looks like it has a lock on it. Could anyone explain what it means and why the files are missing from the backup?

2018-01-23 20_28_13-Duplicati

Thank you.

Update: When selecting just the Downloads folder for backing up it seems that the backup works properly. When adding it as normal to a full system backup all the files are missing.

Could this be just a display bug? Anyway to check this?

Update2: It seem the same behavior happens for another volume which is mounted in another folder. Backing up the folders independently from the drive which currently has them mounted backups up the data, but backing up just the drive containing the folder does not backup up the data in volumes which are mounted in folders and not with drive letters.

Does anyone know any reason for this?

Because it’s a mount point and not a folder? You don’t have a drive mounted as folder, you have a mount point on a folder. It needs to be specifically stated in the backup that this location needs to be backed up.

So for example if I mount a volume on C:\bigdisk, and in Duplicati I select to backup everything from C:\ it won’t back up C:\bigdisk as it isn’t a folder. However if in Duplicati you specify C:\bigdisk it will backup everything on that volume.

Or I’m not understanding what you’re set up is?

You understood 100% correct, but the point is that when selecting C:, the C:\bigdisk mount is also selected by default and should be backed up.

The way it works now every mount point has to be specified individually even though it is already selected when selecting the containing drive.

There seems to be a new issue I also found. The same thing happens for “C:\Users\All Users” which is not a mount point but I get the same behavior on the folder. The lock, and the separate selection of the folder is needed for the files to be processed.

Yeah that’s a quirk in the GUI as Duplicati won’t know whether it’s a folder or a mount point until it actually parses the files.

Are you sure you’re running Duplicati as an admin when this occurs? Close Duplicati, right click on the desktop icon and select run as administrator.

Yes I am sure. I have the app exe set with Administrator rights and it has been installed as a service. Also VSS is working.

I’m pretty sure that icon in your screen shot is a link in a chain (not a lock) which makes me wonder if you are using the --symlink-policy=Follow parameter?

That specifies links should be followed rather than the default setting of Store that says the link should be backed up but NOT the contents to which it points.

(This page has an image of an actual LOCKED folder - Remove Lock Icon on Folder in Windows 7)


I suspect this is a different issue. If you look at your warnings / errors you’ll probably see nothing regarding the Downloads folder as it’s not an error since Duplicati wasn’t told to follow the links. But you probably WILL see an error about being denied access to “All Users”.

This is probably because the account under which Duplicati is running (I assume your login as the Tray Icon) doesn’t have rights to that folder.

If that’s the case, options include:

  • give the user running Duplicati rights to the folder (not really recommended for security reasons, but should work)
  • run Duplicati as a service under an account that DOES have admin (or backup) rights (side benefit, backups will run even if you’re not logged in AND you can use snapshots)

Yes, it actually looks like a link in a chain. As I stated before Duplicati is already running as a service and has been installed with Administrative rights.

symlink-policy is set to store, which is actually not set but it defaults to it.

I have currently set separate backups for the folders with the chain icon (I have found no info in the docs about what that actually means) and they are working, but I have yet to find a solution for doing a single full backup which includes the folders as well, because I see (and there is) no difference in selecting those folders in the full backup and selecting them separately with the same settings. It should make no difference, but there obviously is from Duplicati’s perspective.

It’s like having 2 folders in C:
folder1
folder2 (with chain icon which is a mount point)

If you select them both (or just C:) only folder1 gets backed-up. folder2 is completely missing.

If you select them individually in two separate backups then both backup jobs work.

Sorry, I didn’t catch that in the earlier posts but thanks for clarifying for me.

If you mean the Duplicati docs then you likely won’t find anything about the chain icon as that is not something from Duplicati (most likely Windows, but possibly a third party tool).

Store means store the link, not what the link points to. I suspect if you change it to Follow then it will work as you expect (without needing the specific mount points added to the source list).

1 Like

I have started a backup with symlink-policy=follow and it seems that it now sees the full size including the mounts. It will take around 2 days to finish, so I’ll update back once it’s done. But first impresion is that the issue is solved by JonMikeIV’s previous reply.

1 Like

And it is a win!
Still getting some other errors regarding VSS but nothing important so, the posted issue is solved.

Further feedback:

Source:429.65 GB
Backup:246.66 GB / 2 Versions

  • Full file backup around 1.6M files on Windows Server 2008 R2 Web Edition
    BeginTime: 26.01.2018 20:12:38
    Duration: 00:00:21.1082073
    BackendStatistics:
    RemoteCalls: 1017
    BytesUploaded: 264849968801

Running options:
–snapshot-policy=Required
–no-backend-verification=true
–backup-test-samples=1
–overwrite=true
–skip-file-hash-checks=true
–check-filetime-only=true
–symlink-policy=Follow
–blocksize=1MB
–store-task-config=all

Restoring around 550 MB large files:

FilesRestored: 8
SizeOfRestoredFiles: 619190044
FoldersRestored: 4
SymlinksRestored: 0
FilesPatched: 0
FilesDeleted: 0
FoldersDeleted: 0
SymlinksDeleted: 0
MainOperation: Restore
RecreateDatabaseResults: null
ParsedResult: Warning
EndTime: 26.01.2018 20:51:16
BeginTime: 26.01.2018 20:47:27
Duration: 00:03:48.5340714
Messages: [
Searching backup 0 (25.01.2018 22:50:12) …,
2 remote files are required to restore
]
Warnings: [
The supplied option --store-task-config is not supported and will be ignored <-- UNFORTUNATELY
]
Errors: []
BackendStatistics:
RemoteCalls: 2
BytesUploaded: 0
BytesDownloaded: 1047531434

This is my first successful backup/restore using Duplicati on Windows. So my first conclusions after playing with it for around a week regarding Windows compatibility is split in two:

The dark side:

  • It needs a lot of reading to get all the right information in the software, a nice simple guide/wiki would be a blessing
  • The product is truly beta at the moment, it is in no way adequate for production
  • A lot of work needs to be put in making it at least 50% faster, backing up large volumes with large number of files and small block sizes is a headache
  • Still a lot of errors that need to be looked into for making the product available to the masses, dev team should become larger for proper Q&A
  • The web interface needs a lot of work
  • Bandwidth shaping needs some work

Into the light:

  • For those like me with some higher technical skill and who have no problem or lack of resources for taking chances with beta software, I’d say it has far exceeded my expectations
  • I like the fact that it puts little stress on the systems being backed up, that is generally a problem with classic software, proper thread control should be enforced by default in future versions
  • Restoring the files works better than expected with seemingly complex algorithms behind the software

I really think it is the most promising backup software I have found in a long time, so I have become a paying backer. Gooo Duplicati !

1 Like