Restore all versions, but smart

[f4253a38b0]

Good evening. I hope eveyone enjoyed the holidays.

I may have used duplicati a little bit wrong. I should have seen this ending coming, never thinking about the end. And I may need help disbanding a backup that is no longer needed.

I had duplicati monitor and backup a folder that had a good split of files changing, and staying the same.
Now that folder has no longer to be monitored, but a lot of it is supposed to be archived.
But a good chunk could also be deleted, and I’d like to archive them outside of Duplicati archives.
The problem is, the old versions should be manually sighted.

There are 153 Versions of the backup, which is all in all 112GB big. Could have been better, could have come worse.

My question is:
Is there a way to restore everything duplicati has every archived, but in a smarter way?
Meaning not to restore files again in the next version that were already restored identical already?
I imagine basically repacking everything in the original folder, but with suffixes/präfixes for different dates, if neccessary.
If that is impossible with the included tools, has anyone run into something similar?
How to I “crosscompare” all the versions without needing 10’s of Terabytes of free disk space.

Thank you very much.

[ts678]

@f4253a38b0

If that is impossible with the included tools

It’s good to hear it phrased that way, because this will probably need Duplicati.CommandLine.exe.
You can start with an Export As Command-line then change the backup command to do a restore.
You’ll need to give it a –version, and you’ll likely need to restore versions by hand, or using a script.

@f4253a38b0

the old versions should be manually sighted.

I wasn’t sure what sighted meant, but below I guess you want each version of a file to be its own file.
Versions that have been deleted are of course gone, but even after that you could use a lot of space.

@f4253a38b0

I imagine basically repacking everything in the original folder, but with suffixes/präfixes for different dates, if neccessary.

Above will give you different names, based on current date and possibly an additional number value.
You then need to go through them, I suppose, to figure out which ones to discard and which to keep.

If you don’t like Duplicati’s naming, or wish to do something like embed modification time, tools exist.
You might consider restore to a spot that’s not the original, especially before you have practiced this.

@f4253a38b0

Meaning not to restore files again in the next version that were already restored identical already?

Duplicati does no needless restore when the file is already as desired. This confuses some people…

@f4253a38b0

How to I “crosscompare” all the versions without needing 10’s of Terabytes of free disk space.

The COMPARE command, e.g. in Commandline (adapt backup syntax by putting versions in the box)

[f4253a38b0]

First of all, thank you for answering to my particular problem.

@ts678

because this will probably need Duplicati.CommandLine.exe.

I’m trying to get familiar with the CommandLine.exe. The docs could maybe use a collection of examples with the correct syntax. It can be quite confusig when Apostrophes are used and when not and so on.
As soon as I realiably can restore the version I want to the folder I want with the settings I want, I’ll report back.

In the meanwhile I started a manual process of restoring the full archive, comparing to the consolidated version, deleting identicals, renaming and adding to the consolidated version.
It requires the full restore though and the full comparison, it’s tideous and puts my spinning rust through a lot.

@ts678

I wasn’t sure what sighted meant, but below I guess you want each version of a file to be its own file.

That was apparently a false friend in my vocabulary. I meant that each file, after all would have to be looked at by human eyes to decide if to be deleted or to be archived properly and permanently.
But you are correct, what I basically want is each file and each distinct version of it as and individual file.

@ts678

Above will give you different names, based on current date and possibly an additional number value.
You then need to go through them, I suppose, to figure out which ones to discard and which to keep.

If you don’t like Duplicati’s naming, or wish to do something like embed modification time, tools exist.

I created a Test_Backup with a Test.txt I changed back and forth but with the same filename, to understand the “different versions with timestamp in file name”-option and what it does.

Duplicati created some errors I yet don’t understand.
Partially because the timestamp only contains the day, not the actual time, sometimes I have two versions on the same day.
Partially because that way, if the version I restore first is unique, and all the next ones are different from the first but identical with each other, duplicati would not catch that would it? Meaning two versions of the file, but still 153 files in the consolidated folder. Or is duplicati aware of its timestamps and checks for those as well?

@ts678

You might consider restore to a spot that’s not the original, especially before you have practiced this.

Here I miswrote again. Ofcourse I am restoring not in the “Original”, I meant I want to keep the original-folder structure.

@ts678

The COMPARE command, e.g. in Commandline (adapt backup syntax by putting versions in the box)

This command is new to me. I’ll need a couple of hours and tries to understand it.

[ts678]

@f4253a38b0

It can be quite confusig when Apostrophes are used and when not and so on.

If you mean quoting, that’s why you let “Export As Command-line” set it up automatically for you.
Those quotes are not part of Duplicati syntax, but depend on the particular OS that you’re using.
General Command Prompt rule is double quote things with spaces, to keep the pieces together.
Summary of Important Concepts For Quoting and Escaping Windows Command Line Arguments

@f4253a38b0

Meaning two versions of the file, but still 153 files in the consolidated folder.

Please test for yourself. When I tried, it was not tricked by repeated restore of same file content.

[f4253a38b0]

So I experimented with the commandline and Duplicati’s timestamp function.
And it seems it does recognize it’s own old timestamped files. Which makes this thread basically solved.

I just need a script or something for duplicati to go through all 153 versions of my backup.