Implementing the feature to restore any version of a single file

Developer
1

Is anyone actively working on Show all versions of a file in the recovery tree? I started working on it quite a while ago, made some really good progress, and then had to drop it. If no one is working on it I would like to finish it.

I don’t know if there has been any consensus on what the UI might look like. For my testing I did the simplest thing of having a dropdown for each file on the normal restore screen, similar to this. I didn’t have any columns, just the modified time, a space, and then the file size.

3 Likes
2

I worked on this a while ago and ended up creating a small AutoHotkey script and we’ve been using it for about half a year now.
I’m a ex-CrashPlan user (we had their business offer until last year), so I tried to make the UI have a similar feel and feature set as what CrashPlan has.

I had plans to port this to the web UI, but time was never on my side.
Hit me up if you need help at any point.

1 Like
3

I remember that CrashPlan capability, but I forget how it handled deleted files, e.g.
Restore deleted file (recently active on forum, and I think all of this has old topics)
notes the difficulty of getting started on restore if you can’t find versions to start at.

Deleted file retention used to be one of CrashPlan’s selling points. It got worse but
maybe it left some good ideas behind in terms of UI features handy for that design.

I haven’t thought about this in awhile, but thought I’d chip in another common “ask”.
Maybe that’s a separate request. It sounds like you already have a first cut at a UI.

4

I don’t think anyone is, so if you can pick it back up again that would be fantastic.

I liked the old CrashPlan interface for file restores, and the “show deleted items” was helpful.

1 Like
5

Yeah, it used to be great in CrashPlan and indeed one of the best points.
We used it a handful of times in 10 years, but it was a real nice to have.

Duplicati isn’t so great at handling deleted files, sadly.
I wish it had a specific setting to handle deleted files so that we could keep them for longer.

We workaround this by setting a very long retention for some backups, and we don’t ever prune those sets.

I might add a feature specifically to list deleted files to my script at some point, but since we didn’t use it much in all those years, I never got around to think about it further.

CrashPlan’s UI was not-so-bad, and then it got significantly worse. After their deleted file retention rules changed, I decided to just give up on it. It wasn’t even cheap and for less than I used to pay I have some space on rsync.net (which I trust MUCH more).

Those handful of times we did recover files in CrashPlan, the only thing of notice was just how slow and bloated it was. If you wanted to restore several revisions to find where a specific change happened, you would spend a lot of time restoring each revision separately, and would have to wait quite some time in-between. Not very nice.

One thing I always wanted was to be able multi-select a bunch of revisions and restore them all with timestamps. This way, I can easily diff and find whatever I’m looking for.
This is a bit of a poor-man’s source control, but my wife (the main user of this script) used this a few times with her art work, so it was a useful feature to have, even though she does use source control as well.

6

Excellent, thank you for the offer. I’m glad you’ve been able to use AutoHotkey to get this functionality for a while.

To clarify, is the ask to handle deleted files? I think that should be handled as a separate task as it will have quite a different focus.

Here is the super simple UI I have right now. This is just the regular restore files screen. Every file would have a dropdown. When the user clicks the dropdown, the versions of the file are fetched and displayed like this. The time displayed is the file’s modification time not the backup time.
RestoreVersion

I have tested and am able to restore any version correctly. Right now I am working on what happens when you select multiple versions of a file. My plan is to append the modification date and time to the filename.

There are other questions like should the versions displayed show ALL versions of the file? Even if I select to restore from the first backup? I lean towards yes but then it might be nice to indicate for each version what backup it corresponds to as well. I’m sure there will be a bunch of little details like that, that come up.

2 Likes
7

There is an unlikely potential for version collision if you go to minutes. Might be fine to view but

might be an actual filesystem problem on a name collision. I suggest you check the behavior of
overwrite option (also in GUI buttons) does. I forget exact behavior, localized time handling, etc.

This seems like it could help solve requests like:

Open File from History/See File Content
Can I browse files in larger backups without having to restore the whole thing?

version option claims to allow a syntax where you can say several. I wonder if restore can do it?

EDIT:

Help text says:

  --time=<time>
    Restore files that are older than the specified time.
  --version=<int>
    Restore files from a specific backup.

I recall having had some trouble specifying by time (might be user error). The help text for that is

Duplicati supports absolute and relative dates and times:

  now --> The current time

  1234567890 --> A timestamp, seconds since 1970.

  "2009-03-26T08:30:00+01:00" --> An absolute date and time. You can also use
  the local date and time format of your system like e.g. "01-14-2000" or "01
  jan. 2004".

  Y, M, D, W, h, m, s --> Relative date and time: year, month, day, week,
  hour, minute, second. Example: 2M10D5h is now + 2 months + 10 days + 5
  hours.

and I mention it here because time seems somewhat in the neighborhood of what you’re making.

8

Sorry - I am new here (and I am not a developer) - moved from crash plan 2 months ago.
Viewing deleted items (and restoring them) would be nice.

In CP - there are 2 switches - display deleted and display hidden.

When viewed in the UI - there is clear indication what is what.
Would be really nice to have such feature.

Also the versioning is very positive. What is on the screenshot looks good.
I am not 100% sure but I believe in CP even the deleted items had versions (the UI would replicate for the deleted items).

9

Did you have a big enough backup for the restore display to take time you could notice?
If so, do you recall if the switches made a difference. I worry a bit about Duplicati speed.
Some of the SQL takes awhile, and we periodically get complaints about slow tree view.

10

I had 1.5 TB backup and the consumed space on their server was 2 TB.
There was no noticeable slowdown for showing the list of files when I checked the switches (to the best that I can recall).

If there was slowdown - it was related to the number of files in a folder. In my case I had maybe more than 23000 files in a folder (this is not unique as these are pictures - I am a photographer - my hobby).

The restore was not too fast (regardless if there were deleted files or not) because it took a while to prepare the restore - normally a zip file (if I am not mistaken). Once the zip is created however - it is up to the network.

A significant drawbacks in CP when it comes to restore are

  • If wanted to restore particular type of files (.XMP for example) but not restore (.CR2) - the selection was not elegant.

  • I had a challenge restoring from a particular date (again - selecting particular type of files)

  • I had a challenge selecting only the deleted files

11

That’s interesting. I would think going down to the second should resolve that problem. e.g. “examplefile-20220503_200517.txt”, or however it should be formatted, for examplefile.txt that was modified on May 3rd 2022 at 8:05PM and 17 seconds. But I will look and see what the code for the overwrite option is doing. Thanks for the info.

12

I realize asking the internet for ideas is dangerous because everyone has a different opinion :slight_smile:. Here’s where this stands right now.

Expand a file and you’ll see the versions with the modified time of the file and the size. The bolded version represents the version of the file that is in the selected backup at the top, “Restore from”. What I am referring to as the “default” version. So if I changed the backup to 0, the default would change to the version from 11:20 AM.
image
If the default version is the only version selected it gets restored the same way as every other file right now, and would have the original filename, firstFile.txt in this case.

However, if any single version is selected that is not the default, or if multiple versions are selected, they will always be restored with the file modification date and time appended, like so
image

Which then leads into the fun weird situations. If I then immediately do another restore of the default version for any backup I’ll wind up with three files. because the default will restore with the original filename. In this example I restored the default version for backup 0, the 11:20 AM one. firstFile.txt and firstFile_220619-112016.txt have the exact same contents.
image

And then if I modify 111730.txt, and try to restore that version as non-default (backup 0 was selected) and choose the “Save different versions with timestamp in file name”, I get the backup timestamp appended to the file, which already has the file modification time. Definitely not ideal! haha
image

I was curious what people thought about restoring a version of a file and always no matter what having the file modification time in the filename. That would simplify this some, though there would still need to be changes so the backup modification time is not appended to the filename after the file modification time. Probably doing a simple increment a number instead. 111730-1.txt, 111730-2.txt, etc.

Always having the file modification time might be less than ideal as, for example, someone could do a restore and then immediately run some program that expects a file, but since the file would have a different name the program would not work.

Maybe it should be if a single version is selected to restore, default or not (regardless of what backup is selected) then restore it with the original filename? If overwrite is selected, it would overwrite. If don’t overwrite is selected then put the file modification time in the filename. If don’t overwrite, there’s already a file by that name, then append -1, -2, -3 after the file modified time.
If multiple versions are selected choose the most recent one and restore that with the original filename and all other versions being restored would have the modified time in the filename?

I think there’s some other problematic situations that I can’t think of at the moment.

Sorry for the long post. What would you as the user expect? What would be the most intuitive?

1 Like
13

This would be my preference.

But if the user selects multiple versions to restore, I’m good with appending the timestamp to the filename of each version restored, because there’s no good way to determine which version the user may want to use the original filename.

Awesome work btw!

14

I’ve been thinking about it some more and I think you’re right @drwtsn32. That would probably be the expected behavior and also the easiest to understand, always restore to the original filename if one version is selected (regardless of what backup is selected), otherwise append timestamps. I will head in that direction.

Thanks! As usual, life has gotten in the way but it’s been figuring it out.

1 Like
15

Sir, your work was really wonderful, by the way, how did you get the file’s modification time, before your final work finished, is there any way to get modification time of a certain file for me in command line? I really need to look up the file’s modification time to decide which file I restored

16

If you really mean it’s already restored, just look at the file. If this is planning, do you know a rough time?
This is crude, but you should be able to restore a few versions around then until you find what you want.
This will also let you look inside the file, if that helps decide, or is an exact time the only deciding factor?

If “really need” gives you motivation to do hard work, manual database browsing can get you an answer.
This would need more detailed guidance posted here, along with a tool such as DB Browser for SQLite.

17

I am sorry I didn’t describe clearly, I want to look up the file’s modification time in the restore files interface to decide which version to restore, now I must restore all the file in different versions, then look up the modification time to find out which one is needed. And I also didn’t find the function to show modification time in the Command Line Tools.

18

Why “all the file”? Or do you mean the single file? How many different versions are time-based candidates?
This gets back to the question of why not start somewhere around the right time and restore some nearby?

I don’t think there is one, and I think the new change is for GUI. Why is Command Line a hard requirement?

EDIT:

If there were such an option (but I don’t think there is), it would be on The FIND command also called list.
What you might be looking for is the ability to give it a path and have its listing show dates as well as sizes:

Listing files and versions:
C:\backup source\short.txt
0       : 9/28/2022 11:07:17 AM 58 bytes
1       : 9/28/2022 11:06:01 AM 54 bytes
2       : 9/19/2022 10:47:55 AM  -

This feature doesn’t exist AFAIK, so you need to find another way. Maybe you can describe exact use case. Having timestamps would still need you to know the desired time or guess. Do you know the desired time? Generally the first backup after that time is the one you want, but you can inspect one before it to be sure…