Hi, if say I backed up a Linux NAS using Duplicati and thereafter I want to change the OS of the NAS to Windows, is it possible to continue the same backup jobs? Or do I have to reupload all files again?
Edit: To be exact, how can I avoid the “The backup contains files that belong to another operating system” message without also having to reupload all data (and even rebuilding the database)?
There are no options that I know of to do this. The design handles pathnames (which differ) and metadata such as time stamps and owner/permission information (which differs), so it’s not just an arbitrary refusal.
One bit of good news is you won’t be doing a database Recreate (if that’s what you spoke about) because backing up again will create it again. Interestingly, people who lose the database sometimes say it’s faster.
Losing old versions may be a concern. Maybe you could keep Linux (even Windows Subsystem for Linux) around for that need, or possibly even for running Linux Duplicati on WSL. Not sure if that’s been tried yet, and there are some issues to WSL and the Windows it’s in getting along, but it seems to be getting better.
How to Access Linux Files in a Windows Subsystem for Linux (WSL) Distro from Windows 10
Cross platform restore not possible
The backup contains files that belong to another operating system
Thanks! I am ok with keeping the old Linux filesets and even the database in another folder (maybe even away from the remote storage completely)
As long there is a method where I do not need to reupload the data especially the heavy dblock files I am ok.
May I know what happens when the remote storage has no fileset (i.e. dblock and dindex files) files?
“There are no options that I know of to do this” meant no Duplicati option switches, so maybe backup from scratch, which creates the database (so no need for Recreate button which probably isn’t cross-platform).
Optionally, you make the Linux backup available via Linux-like system if someone really wants old versions. Beyond WSL, a separate Linux system or even a VM (e.g. VirtualBox) on Windows might give you enough.
The new remote storage being empty is the normal state for a new backup, so all that goes as usual. The size of upload will probably be less than the old backup (assuming the old one has saved some versions).
One possible option beyond Duplicati is to run Linux Duplicati in WSL, if you have the right Windows and it all works. I haven’t used it at all, but the below article implies that access from WSL to Linux works alright, and so maybe with some symbolic link work you can make your old Linux path refer to new Windows area.
Frequently Asked Questions about Windows Subsystem for Linux
For example, your
C:drive is mounted under/mnt/c
So if Linux used /mnt/source_files you might do ln -s /mnt/c/source_files /mnt/source_files, so old paths still work. WSL seems to do some sort of permission conversion. Not sure how well, or if this plan stands much chance of working. Possibly someone in the forum has tried WSL and I didn’t notice it…
Possibly you’ll be breaking lots of new ground. First thing to web-research is whether mono works on WSL. More conservative from Duplicati view might be a Linux VM treating Windows backup as a shared folder or something. More exotic might be Linux on Windows via Docker for Windows, but I know very little Docker.
Docker on windows vs Docker on Linux has one amusing discussion about what’s sane and what’s crazy. Certainly a clean start is the proven path, but feel free (if you like) to see if you can find a way to avoid that.
Hmm I see, I misread your post earlier which is why I edited my question a bit (not sure if you have seen the edit yet)
For backing up, am I right to say that the only way is to reupload the data set after the OS change? I asked as I saw this from Changing source OS - #4 by John which sounded like what I hypothetically want to try
As for restoring, strangely enough all I needed to do was to copy the database and settings of the backup job from the Debian machine to the Windows machine, import this backup job, check that the location of the remote data destination is correct (or relocate it to a local HDD if I have already downloaded them) and then restore the files as per usual.
I saw it, found code, thought avoiding that message would need hacking on paths in the database, and didn’t think that was a thing you’d want to get into, or that I could help with. I also considered noting that material could be reused if you (or someone someday) could craft code to convert and repackage it all. The data blocks of the file contents should be OS-independent – but that’s not all that’s in the backup…
Again, when you say “only way”, it’s up to you to decide how ambitious you want to be in finding another.
I suggested multiple unproven ways to avoid that by continuing to use a Linux environment on Windows, however you have found another unproven idea involving hacking your database by hacking backup files then using them to recreate the database. This seems slightly more appealing than direct hacking of DB although we never heard if it worked, and I still wonder what happens to the Windows metadata stored in remote files. You might have some info about that from what happened on your surprising cross-platform restore. If it somehow was able to get a creation and modification time onto Windows, that’s a good sign.
I’m not sure I fully follow the “Changing source OS” steps, e.g. what is done with current dlists that aren’t the dummy? If you recreate the database, it will need the dlists to restore that version, meaning they may all need to get hacked. If you can persuade Duplicati to pull in all of the dblocks then not compact data as unreferenced source blocks (maybe set –no-auto-compact, just in case), then the next backup might put them to use again if they are still relevant, i.e. useful as a block in one of the current files being backed up. This would avoid reuploading all the data for the current version. Old versions could use Linux, if need be. Allowing or forcing compact would probably trim your remote space usage whenever you’re ready to do it.
Seeing the lead author suggest this raises confidence it probably works at least in part, but I can’t detail it.
EDIT: If you conclude mass dlist editing is needed (I don’t know) to convert your old dlist files to Windows, and if you don’t want to use the Python or other code-heavy method, and because of the mentioned issue with forward slashes being present in hashes, a possible alternative might be edits by regular expression. RE syntax varies, but if you use vim for the dlist edit, it’s said to be similar to Perl, so I think that’s enough.
How to escape special characters in building a JSON string? shows special cases it might need to handle including the most guaranteed-relevant case which is that Linux forward slash becomes two backslashes on Windows. If it’s already encoded as the backslash and forward slash, then that becomes the backslash backslash. You can look. You probably would also add your Windows drive letter in front of path. The main goal of the regular expression is to ensure context, e.g. verifying "path": in front of the string being edited. This isn’t the “proper” way to deal with JSON, but if the proper way is too hard, it might give you enough…
Hmm. I was thinking along the lines of “how easy can this be done by an end-user within the GUI?” when I was asking/phrasing about these. I sadly haven’t have time to test what you mentioned (the part about modifying database/dlist files seem most probable) but it seems to me using VMs (or even a separate physical machine) may be the most data-safe option at this point. My gut feeling tells me that for most datasets, it is much faster to simply upload a new set from the new OS rather than trying to modify all of these files…
As for metadata; so far for all cross-platform restores I did not use the “restore permissions” option. This led to files being restored that I can use right away, along with their modified dates being restored too (not sure about the creation date as I didn’t notice).