Cloud drive integration (files on-demand)

Some users are starting to run into warnings related to Duplicati trying to back up integrated cloud drives that use “files on-demand” (so basically a local stub points to the actual file that lives in the cloud).

Assuming this trend is likely to increase, I’m curious what people think about how to handle these. Should we ignore them, back up the stubs, pull down the on-demand file and back that up, let a setting determine which of those, or something else completely?

When considering this, keep in mind the Duplicati big picture - for example, if we back up a stub then the user deletes the file from the cloud what happens when we restore the stub (assuming the OS will even allow us to). Similarly, if we back up the cloud file itself (instead of the stub) then when we go to restore it do we overwrite the stub and let the cloud sync tool handle recreating the stub?

Your thoughts are requested, but if you just want to click buttons - here’s an informal poll. :slight_smile:

  • Ignore them (don’t back up)
  • Back up the stubs
  • Pull down the full file and back that up
  • Use a setting so the user can determine which of the these happens on each job
  • Something else
0 voters

I voted for the use a setting, but actually my standard would be

  • Pull down the full file if metadata suggest (e.g. file time changed) or file isn’t in backup yet
  • Backup the file
  • Restore the on-demand flag so local space is freed up again.
1 Like

Hi @tomsedi, welcome to the forum - and thanks for the input!

Hi @JonMikelV,

I think the user should choose what is best for his individual situation.
The most usual setting might be to ignore the stubs.

That’s because the user does already makes the decision to download or not to download in OneDrive. So not downloading a folder via OneDrive and then pulling the content down via duplicati doesn’t make too much sense. But there might be use cases where the OneDrive backup should be guaranteed to be complete. So having a switch to activate that would be great although not neccessarily in the first step.

Just ignoring the stubs is a simple and stable choice without the complexity that might arise with restored stubs.

And as you stated “this trend is likely to increase” so a solution that would allow backups of local OneDrive Caches without all theses warnings would be highly desirable :slight_smile:

Cheers
Klaus

See GitHub Duplicati downloads OneDrive online only files #3411 for a possible way to ignore the stubs or not per user preference, using –exclude-files-attributes. Testing has not yet been done, so anyone feel free to try.

I would be inclined to back up the full file (though perhaps with a warning).
The users already have an option; exclude the directories that are connected to the cloud.
I don’t think backing up the stub does anybody much good (in what cases would restoring it be useful? That the cloud solution wouldn’t provide anyway).
So, if the user has decided to have duplicati back up these files, it must be because they want to keep historical versions of them, so we should do it!

5 posts were split to a new topic: Exclude offline files broken with OneDrive?

cross posting from above, doesn’t seem like exclude-file-attributes with Offline worked (files still download during duplicati run)

2 posts were merged into an existing topic: Exclude offline files broken with OneDrive?