Unable to complete restore over NFS - Duplicati.Commandline.Recovery.exe

[Bungeetaco]

@ts678

Losing track of systems. Is the backup server the one whose drives failed, taking out not only the source files but the Duplicati database? In what ways is it a backup server? Is there another backup going to it?

There’s only one box. This box was running a VM, and in this VM, Duplicati ran. This was the backup server in question. The drives where this installation was stored died. I then got new drives to replace the ones that died, and those drives are connected to the same machine that my backup server used to run on.

The way I had it setup, all my other VMs, including my fileserver, were available over NFS, and were mounted on the Duplicati VM. Duplicati saw these mounted folders as local ones and would back them up on a regular basis.

@ts678

One reason I ask is because another option to try is Independent restore program in Python. How’s your Python? Does it look like this restore timestamps or just files, and how essential are the file timestamps? Duplicati backup files don’t care about their timestamps, but I don’t know whether your backup will care.

We’re getting into some rarely used things. Possibly your open file problem is only in RecoveryTool, and regular Duplicati Restore from GUI or CLI would work – if you could get installed and if other things held.

I did try the script you linked during my troubleshooting. Unfortunately I was not able to get it to work on two different systems (Linux and Windows), as it seems the modules have been updated and do not get recognized by the script anymore:

C:\Users\r00t>pip3 install pyAesCrypt
Collecting pyAesCrypt
Downloading https://files.pythonhosted.org/packages/c5/99/ebae89da08a443d0191eff631d66b71955eec7e370e2a309d85e499ca18a/pyAesCrypt-0.4.3-py3-none-any.whl
Collecting cryptography
Downloading https://files.pythonhosted.org/packages/a4/9c/9ad11ad293bb75d37fa63a0d1f47ae7f6ccda32bc7452bf2ffb788ca753f/cryptography-2.8-cp37-cp37m-win_amd64.whl (1.5MB)
|████████████████████████████████| 1.5MB 1.3MB/s
Collecting cffi!=1.11.3,>=1.8
Downloading https://files.pythonhosted.org/packages/71/71/6011c17e8a523be0be20149ed7ba7e8671db7ec6fc52868e4f9a49d527f8/cffi-1.13.2-cp37-cp37m-win_amd64.whl (175kB)
|████████████████████████████████| 184kB 3.3MB/s
Requirement already satisfied: six>=1.4.1 in c:\users\gabriel\appdata\local\programs\python\python37\lib\site-packages (from cryptography->pyAesCrypt) (1.12.0)
Collecting pycparser
Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB)
|████████████████████████████████| 163kB 6.4MB/s
Installing collected packages: pycparser, cffi, cryptography, pyAesCrypt
Running setup.py install for pycparser … done
Successfully installed cffi-1.13.2 cryptography-2.8 pyAesCrypt-0.4.3 pycparser-2.19

C:\Users\r00t>pip3 install ijson
Collecting ijson
Downloading https://files.pythonhosted.org/packages/bb/f9/9dd57fe468e033610b979adcb45b159b76309ea4df63f70a10cfb75d71ae/ijson-2.5.1.tar.gz
Installing collected packages: ijson
Running setup.py install for ijson … done
Successfully installed ijson-2.5.1

Z:\Duplicati>python3 ./restore_from_python.py
Traceback (most recent call last):
File “./restore_from_python.py”, line 14, in
import ijson
ModuleNotFoundError: No module named ‘ijson’

Z:\Duplicati>pip install ijson
Requirement already satisfied: ijson in c:\users\gabriel\appdata\local\programs\python\python37\lib\site-packages (2.5.1)

Z:\Duplicati>pip install pyaescrypt
Requirement already satisfied: pyaescrypt in c:\users\gabriel\appdata\local\programs\python\python37\lib\site-packages (0.4.3)
Requirement already satisfied: cryptography in c:\users\gabriel\appdata\local\programs\python\python37\lib\site-packages (from pyaescrypt) (2.8)
Requirement already satisfied: six>=1.4.1 in c:\users\gabriel\appdata\local\programs\python\python37\lib\site-packages (from cryptography-pyaescrypt) (1.12.0)
Requirement already satisfied: cffi!=1.11.3,>=1.8 in c:\users\gabriel\appdata\local\programs\python\python37\lib\site-packages (from cryptography-pyaescrypt) (1.13.2)
Requirement already satisfied: pycparser in c:\users\gabriel\appdata\local\programs\python\python37\lib\site-packages (from cffi!=1.11.3,>=1.8->cryptography->pyaescrypt) (2.19)

Z:\Duplicati>python ./restore_from_python.py
Traceback (most recent call last):
File “./restore_from_python.py”, line 23, in
from pyaescrypt import pyAesCryptDecrypt, fail_with_msg
ModuleNotFoundError: No module named ‘pyaescrypt’

I even made sure to try both python and python3, as the pyaescrypt module uses Python 3 from what I could gather.

@ts678

Keeping NFS in the mix adds potentially more questions. Can’t most VM system share host files with the VM in some limited way (I guess you want to be super-sure nothing you do breaks your only download)?

Unfortunately no, this is a VM, thus it doesn’t have any visibility over the host.

@ts678

But the latest approach sounds like a Windows PC with access to backup files via either Samba or USB.

That’s correct, so I now have a 10TB drive that is plugged into my desktop PC running Windows that was previously mirroring the data contained on the other drive on my host containing the Duplicati backup.

@ts678

Is it only dblock files? Regular operations also need dlist and dindex. I think RecoveryTool does without dindex, but needs dlist. There’s also a step in directions about building an index file RecoveryTool-style. There was talk earlier of index, so I suspect there’s an index, but it wouldn’t hurt to verify what’s there…

I do have a dlist.zip file for each individual backup dates, but I do not have any dindex.zip files. I checked my Backblaze storage to see if I could find the dindex files in there, but I couldn’t, so it seems they never were created during backup. I did however generate the index.txt file in the process of using the RecoveryTool.

@ts678

Is there any other context output before that to show where this was?

Yes, the console did push out the same exact output as you quoted. It took about 5-10 minutes for the restore task to actually start.

@ts678

Your command taken in isolation looks to me as documented, but I haven’t set up a test. First, does the output you get leading up to the error look sane for your situation based on the example shown above?

Because you’re doing a cross-platform restore (original has forward slashes, restore has backslashes), I’m wondering whether that difference could be tripping things up. Regular GUI restore can deal with it, somewhat, but IIRC it won’t let you restore to original file paths – you have to point it to a restore folder.

Even if that was the case, since I’m doing it over Windows, the path is using the same type of slashes as the rest of the filesystem as shown below. I don’t know how that translates when the program actually runs, I’m not a programmer.

https://rayquaza.bungee.systems/s/jXDc9yzrPPMt22N/preview

I’m very confused.