Local folder restore fails with "Failed to connect: No filesets found on remote target"

chasingmaxwell · January 6, 2023, 3:17am

There are a few other similar topics, but none of them seem to address my specific situation. I’m trying to restore some files from an old computer to a new one and keep running into the error “No filesets found on remote target”. I’m restoring from a local folder containing duplicati-{hash}.dindex.zip.aes files and duplicati-{hash}.dblock.zip.aes files. It contains 846 files and is 21.4 GB. It started as a backup for my old laptop stored in an s3 bucket, but when I had this issue on s3 I downloaded the bucket to a local folder hoping to take connection issues out of the equation.

Here’s what I’m doing:

Navigate in the Duplicati UI to the “Restore” page
Select “Direct restore from backup files…”
Press “Next >”
Select Storage Type" of “Local folder or drive”
Select the “Folder path” containing my backup
Press “Next >”
Enter the encryption passphrase for my backup
Press “Connect”

Then I receive the error: “No filesets found on remote target”.

From my logs I see:

Jan 5, 2023 10:03 PM: Failed while executing "List" with id: cecd1a1c-f117-4e6a-9399-20997e9965f6
Duplicati.Library.Interface.UserInformationException: No filesets found on remote target
  at Duplicati.Library.Main.Operation.ListFilesHandler.Run (System.Collections.Generic.IEnumerable`1[T] filterstrings, Duplicati.Library.Utility.IFilter compositefilter) [0x002dc] in <7201532dcc0443468ec0ba778f89f3ac>:0 
  at Duplicati.Library.Main.Controller+<>c__DisplayClass21_0.<List>b__0 (Duplicati.Library.Main.ListResults result) [0x0001c] in <7201532dcc0443468ec0ba778f89f3ac>:0 
  at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String[]& paths, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x0026f] in <7201532dcc0443468ec0ba778f89f3ac>:0 
  at Duplicati.Library.Main.Controller.RunAction[T] (T result, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x00007] in <7201532dcc0443468ec0ba778f89f3ac>:0 
  at Duplicati.Library.Main.Controller.List (System.Collections.Generic.IEnumerable`1[T] filterstrings, Duplicati.Library.Utility.IFilter filter) [0x00021] in <7201532dcc0443468ec0ba778f89f3ac>:0 
  at Duplicati.Server.Runner.Run (Duplicati.Server.Runner+IRunnerData data, System.Boolean fromQueue) [0x00374] in <59054a017605435993aba9f724246795>:0 

Jan 5, 2023 10:03 PM: Request for http://localhost:8200/api/v1/backup/cecd1a1c-f117-4e6a-9399-20997e9965f6/filesets gave error
Duplicati.Library.Interface.UserInformationException: No filesets found on remote target
  at Duplicati.Library.Main.Operation.ListFilesHandler.Run (System.Collections.Generic.IEnumerable`1[T] filterstrings, Duplicati.Library.Utility.IFilter compositefilter) [0x002dc] in <7201532dcc0443468ec0ba778f89f3ac>:0 
  at Duplicati.Library.Main.Controller+<>c__DisplayClass21_0.<List>b__0 (Duplicati.Library.Main.ListResults result) [0x0001c] in <7201532dcc0443468ec0ba778f89f3ac>:0 
  at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String[]& paths, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x0026f] in <7201532dcc0443468ec0ba778f89f3ac>:0 
  at Duplicati.Library.Main.Controller.RunAction[T] (T result, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x00007] in <7201532dcc0443468ec0ba778f89f3ac>:0 
  at Duplicati.Library.Main.Controller.List (System.Collections.Generic.IEnumerable`1[T] filterstrings, Duplicati.Library.Utility.IFilter filter) [0x00021] in <7201532dcc0443468ec0ba778f89f3ac>:0 
  at Duplicati.Server.Runner.Run (Duplicati.Server.Runner+IRunnerData data, System.Boolean fromQueue) [0x00668] in <59054a017605435993aba9f724246795>:0 
  at Duplicati.Server.WebServer.RESTMethods.Backup.ListFileSets (Duplicati.Server.Serialization.Interface.IBackup backup, Duplicati.Server.WebServer.RESTMethods.RequestInfo info) [0x000a4] in <59054a017605435993aba9f724246795>:0 
  at Duplicati.Server.WebServer.RESTMethods.Backup.GET (System.String key, Duplicati.Server.WebServer.RESTMethods.RequestInfo info) [0x001f6] in <59054a017605435993aba9f724246795>:0 
  at Duplicati.Server.WebServer.RESTHandler.DoProcess (Duplicati.Server.WebServer.RESTMethods.RequestInfo info, System.String method, System.String module, System.String key) [0x00146] in <59054a017605435993aba9f724246795>:0

Jan 5, 2023 10:03 PM: Reporting error gave error
System.ObjectDisposedException: Can not write to a closed TextWriter.
  at System.IO.StreamWriter.Flush (System.Boolean flushStream, System.Boolean flushEncoder) [0x00008] in <282c4228012f4f3d96bdf0f2b2dea837>:0 
  at System.IO.StreamWriter.Flush () [0x00006] in <282c4228012f4f3d96bdf0f2b2dea837>:0 
  at Duplicati.Server.WebServer.RESTHandler.DoProcess (Duplicati.Server.WebServer.RESTMethods.RequestInfo info, System.String method, System.String module, System.String key) [0x003bc] in <59054a017605435993aba9f724246795>:0

Some system properties that might be relevant:

APIVersion : 1
PasswordPlaceholder : **********
ServerVersion : 2.0.6.104
ServerVersionName : - 2.0.6.104_canary_2022-06-15
ServerVersionType : Canary
StartedBy : Server
BaseVersionName : 2.0.6.104_canary_2022-06-15
DefaultUpdateChannel : Canary
DefaultUsageReportLevel : Information
ServerTime : 2023-01-05T21:50:37.944907-05:00
OSType : Linux
DirectorySeparator : /
PathSeparator : :
CaseSensitiveFilesystem : true
MonoVersion : 6.12.0

I’m kind of at a loss for what to try next. I hope someone can help!

gpatel-fr · January 6, 2023, 9:03am

Hello

can you check with a single .aes file that you can decrypt it ?

SharpAESCrypt.exe d yoursecretphrase1234 c:\duplicati-20220916T100000Z.dlist.zip.aes c:\duplicati-20220916T100000Z.dlist.zip

ts678 · January 6, 2023, 5:16pm

Welcome to the forum @chasingmaxwell

Any dlist files on S3 or local? Example name: duplicati-20220403T013212Z.dlist.zip.aes.

These are likely what it means by fileset. They describe the files in that backup version.
The dblock contain blocks of the files, and dindex say which blocks are in a given dblock.

How the backup process works
How the restore process works

I hope you find a suitable dlist. Without that, there’s no record of what the files contained,
unless perhaps you can find an old database from where it was stored on the old computer.
If you do have an old database, moving it is a faster way than having to recreate a new one.

chasingmaxwell · January 6, 2023, 6:42pm

Is there a way for me to do this on a linux machine? I tried this:

openssl enc -d -aes-256-cbc -in duplicati-{hash}.dblock.zip.aes -out duplicati-{hash}.dblock.zip

But I get the error “bad magic number”. I hope that doesn’t mean my password is incorrect. From some quick googling it looks like maybe it’s only that openssl isn’t expecting the same format.

chasingmaxwell · January 6, 2023, 6:46pm

I don’t have any dlist files in the local folder or s3, unfortunately. And I lost my database when I was trying to solve this problem by deleting and repairing (I know. Stupid mistake). Is there any method at all without dlist files or an old database for retrieving data from the backup?

Also, just so I can understand how I got in this state, what would cause my dlist files to go missing? I never made any manual changes to my s3 bucket or to the local folder.

gpatel-fr · January 6, 2023, 7:13pm

prefix the command with ‘mono’. It should be available on the computer where you installed Duplicati. SharpAESCrypt.exe is in the Duplicati directory (probably /usr/lib/duplicati)

ts678 · January 6, 2023, 8:12pm

On old computer, did you ever restore, or notice how many versions were on the home screen, e.g.

If you never finished a backup, that would do it. You could also notice by checking logs, emails, etc.
Deleting all dlist files by themselves is hard by design. You need to delete with allow-full-removal:

By default, the last fileset cannot be removed. This is a safeguard to make sure that all remote data is not deleted by a configuration mistake. Use this flag to disable that protection, such that all filesets can be deleted.

If you have historical records from Duplicati or S3, they might reveal things. Were dlist ever there?
If they were once there, the same records might show delete (manual or otherwise). If not, no guess.

In addition to the job database with random letters in name. Duplicati-server.sqlite is server database.
That’s where home screen stats are from, and they’re kind of sticky, maybe updated only by backups.
You might still be able to see something similar to image I posted, showing your old backup statistics.

The best you can do is look over the (default blocksize) 100 KB blocks, assuming dblock files decrypt.
Reassembly directions for what blocks should go into what source files are what the dlist contained.

Restore Backup with only dblock and dindex files is another opinion with a happy ending thanks to DB.

How many jobs do you have? Sometimes Duplicati backs up the database, e.g. at a DB upgrade time. Although it will probably be stale, and not matching current backup, it might contain some prior history.
If you’re a very good coder (like person above with happy ending), you might be able to get some files. Basically, see what else you have in the old database directory besides current job DB and server DB.

EDIT:

Did you ever do a command line backup? Those create databases that might be independent of GUI’s.

gpatel-fr · January 6, 2023, 9:05pm

did you set rules on your backend to delete automatically old files ?
If yes, that’s unfortunate because Duplicati relies on the idea of ‘dumb’ backend, no Duplicati backend should changes files by itself.
If not, maybe it’s a rights problem. Try to connect as manager or something like that. I have seen posts that were talking about such problems with S3.

ts678 · March 4, 2023, 12:37am

There’s now a prototype if you want to see if it works on your dblock and dindex files, if still around…

Using recovery tool with missing dlist files has notes on how ugly recovery can be, but it’s something.