Hi!
I’ve been backing up a few hundreg Gigs so far but stopped in between so the computer doesnt run 24/7. Which means now I have 9 different versions (so far) of the same data (nothing got backuped multiple times as far as I can tell, which is good)
In the end, when the whole backup process is done, will those versions be merged into 1 version so it makes easier to restore files as necessary, or do I have to look up each version when I want to restore files from it?
Thanks
If the most recent backup version contains all data (you can check this by selecting “Restore files” and browse through your backed up files and folders), you can delete all other versions that were aborted (versions should have a suffix partial) using the delete command.
Note that files could have been changed between partial backups, deleting backup parts will result in keeping the most recent version only.
The most recent backup version does NOT contain all data. It only contains the data from that backup session.
Example:
Version 1 contains
E:/somefolder/somefile.txt
Version 2
F:/someotherfolder/someotherfile.txt
Version 3 (most recent)
G:/foo/bar.txt
If I wait now for version 4 to be completed and then delete version 1 through 3, will those files/folders be removed or not?
If the most recent version contains all data that should be backed up (probably the next completed backup), older versions could be safely deleted.
After an interrupted backup, Duplicati generates a synthetic filelist, based on the source files of the previous backup and will exclude unchanged files in the previous interrupted backup(s), hence the suffix partial. These partial backups may contain files that were modified in the meantime, these modifications will be lost after deleting the partials.
Ok I will just wait for 1 complete backup and then look if everything is there.
Sorry, english is not my main tongue and I might not understand you correctly.
I see the partial backups, but I was just thinking if 1 complete backup is done, if they will be “merged” (database wise) to 1, or if every stay partial forever.
This seems strange. It should scan the same files in the same order, but realize that the earlier ones are backed up, so those won’t upload any actual data. I just tested this, and 2nd partial had files from the 1st.
How did you do the stop? The most graceful way is to use the stop button in status bar, then Stop after current file. This is a clean enough stop that the synthetic filelist isn’t needed, which is good because 2.0.5.104_canary_2020-03-25 has a fix (not yet in a Beta) for a problem that could prevent it from working:
Fixed storing
dlistfile after interrupted backup, thanks @seantempleton
Is the destination showing files with dblock, dindex, and dlist for every partial backup? How big are dblock?
Are you using any special options on Options screen 5 that might influence how the file scanning is done?
Ok maybe I misread that restore screen terribly wrong.
The most recent version does have “all” files. But I guess until its fully uploaded (atleast) once I shouldnt delete those partial uploads - which would mean they are gone, is that correct?
In hindsight, it just confused me with those different versions because I interrupted that process so often - I would assume it would try to finish version 1 of the “full” backup to then start its increment process. Now I am left with smaller “full” versions (which in itself are seperate from one another?) (but infact are smaller, non complete version, because I stopped it) - if that makes sense?!
All files so far (including ones backed up earlier) is what I’d expect, and what I’m seeing in my testing.
As far as I know, versions can be deleted independently. Multiple versions may contain the same files, however file data is not deleted until no version needs it, so if latest has earlier, earlier can be deleted, however there’s not really any need to, as the older version likely has little data that doesn’t stay used.
These concepts don’t exist now in the traditional way. Block-based storage engine explains the change.
How the backup process works explains the current details including block-based deduplication design.
What you should have are increasingly large, continuously growing lists of files that appear standalone, though they presumably share file data blocks because they share some of the same unchanging files.
Whenever you finally make it all the way to the end without having to interrupt, it will no longer be partial.
Backup did go through now.
Is there a way to check how much space each “version” is used?
You said “files dont go away until no version needs it” - so I guess its safe to delete ALL version (except the one from nov 6 ?)
Not directly because space use is shared among versions. As mentioned, a version in your case is basically just a file listing (assuming all later versions were exact supersets), so you could go to the destination and look at the dlist file size for the date in UTC you care about. It’s probably quite small.
For other numbers, Complete log of the job log has BytesUploaded which is the progress it made.
The FIND command run in Commandline with Commandline arguments clear shows source sizes.
Should be fine unless you want an old version of a changed file, just like for any version retention plan. Unless you’re opting to keep all versions, some of those old partials may eventually get deleted during normal backup retention processing as chosen on Options screen 5. Or you can delete, if you prefer.