Server Crashed. Was able to copy the database/configs off of the server to another machine.
When I try to do a restore, it says “Connection Failed No Filesets found.” But there’s nearly 40GB of Data.
I have downloaded the set from B2, and attempted a restore that way. Same error.
I have tried decrypting the fileset, same error. I have tried reindexing the fileset. It “reindexes” but then gives the same error when I try to restore.
I did a test before I started using it in production. The tests worked fine. NOw that I need the data, it won’t let me have it.
(Have never used Duplicati 1.x) Have the duplicati-.dblock.zip.aes, and duplicati-.dindex.zip.aes, but no duplicati-*.dlist.zip.aes files.
Anything that can help would be great. I have been fighting this for 5 days now. (If the info I need is available, I have not stumbled across it in my searches.) All other similar topics having the same error message are either dealing with duplicate directory names, or wrong versions of Duplicati. I REALLY need this data. Please, I hope someone has a workable solution.
(I have tried things with the tools, but am willing to try again, in case the steps i followed were missing some important piece.)
Manually downloading all the B2 files to a local folder should work, but it sounds like you didn’t get everything. There should definitely be dlist files in there - one for each backup version. Can you double check?
There are zero dlist files in the backup set. Just checked again. Just dblock and dindex.
If what you say is true, then my backup set is useless?
Rant follows:
So far I am unimpressed with duplicati. It’s not backing up on many of my servers. It just quits without throwing an error. And it won’t restart until you manually restart the service, log in to the gui, and resubmit the job. What’s the point? I may as well not have any backup solution. It only works when you baby-sit it.
I am beyond frustrated right now, as since there’s almost no useful error reporting. (example, when it was quitting initially due to CAPS, watching the actual packets, b2 was indicating that, but duplicati said it was that the program “ended unexpectedly”. After removing the CAPS on b2, I have 1 server running duplicati that is able to do a weeks worth of backups before duplicati just quits. The rest don’t ever complete a backup. I have one site that we throttled duplicati because they don’t have a very big pipe available, and no service providers that can provide one in the area. It still hogs the whole pipe and ignores the throttle settings. I went with Duplicati, because smaller backup sets testing worked flawlessly. But it appears anything over 1GB is out of the question.
I’m sure that if the Log would actually LOG the errors correctly, I might be able to resolve most of them.
Just checked the log on another server. Duplicati is trying to backup files that are in the exclusion list, and hits so many of them that it crashes.
One of the files: C:\Users\Username\AppData\Local\Duplicati\vlock_2
So Duplcati doesn’t even know to exclude iteself? I’ve excluded the AppData Folders, with exception of specific folders in the AppData set, where some applications store data. Duplicati was not a folder I was selecting to backup. Backup software should not crash because a file is unavailable and in use. It should just log the error, and move on.
Backup software should not just quit, and not run the next run just because the previous attempt failed. (especially in the scenario above.) If it’s scheduled to run, it should run. If it’s getting errors, log the errors, and send them someplace.
Never could get the email stuff to work. Had to use Duplicati-Monitoring to be able to get reports.
If there’s no way to restore a backup without the dlist files, I’m screwed.
Almost everything you have mentioned should not be a problem and can be addressed, but for the moment let’s focus on the immediate issue: missing DLIST files. I understand they are missing from the local copy of the B2 bucket, but are they also missing on the B2 side?
Ok, if you have an in-tact local database, the DLIST files can be regenerated. You will be in trouble if you have lost both the job-specific local database AND the DLIST files though.
From what you said in your original post, you have a copy of the local database. This is the sqlite that has random name (letters/numbers) - not Duplicati-server.sqlite. Can you confirm you have the job-specific database from the crashed server?
I was just testing this on a backup that never did the first one. Is that the case here – did one ever finish? Seems like duplicati-monitoring should have noticed a non-finish. Status of backup is on home page too.
Regardless, dlist recreate doesn’t seem to work if initial didn’t finish, so if you try it, use a database copy.
Editing the database Remotevolume table State to “Uploaded” for the dlist file “semed” to trick it to work, however if the backup didn’t finish, there are probably some files missing. Still, might get some files back.
Without a dlist file, this and your backup are about all you have, so let’s keep them safe during experiment.
EDIT: I think it was Uploaded I used (edited), but it would be worth testing again, plus need @mwray input.
Not sure, as I was still testing Duplicati-Monitoring on Other sites, this one was not on it yet.
Lots of files marked uploaded/verified. However, Lots marked uploading/deleting.
So, I need to mark them all Uploaded?
You’re in kind of an unknown state, but it sounds like you have the DB open (but not your only one I hope).
If you didn’t finish any backup, Remotevolume table will (it seems) have a dlist in State Temporary which experimentally gets deleted (along with the maybe one-and-only never-finished version) when you Repair.
Tricking it into thinking it finished and Uploaded the file seems to avoid this, so Repair uploaded a dlist file, which is what’s often done if the destination happens to lose one after it’s been there awhile. Yours might never have gotten there. Do you see any other dlists or just one dlist which never left Temporary state?
Generally, the files on a finished backup will be Verified. Basically let’s see where you are before fiddling. The difference between Uploaded and Verified is that they’re not Verified until they’ve been seen in a listing.
You can look in your Fileset table to see if you only have 1 entry (maybe never done backing up), and you can see roughly how far you got in the File table. But none of this can get anywhere without a dlist upload.
As noted before, take care of your originals. If experimenting, use a copy of DB and a copy of backup files.
I have a ghost of the machine, which is where I got the DB from. The Duplicati info was on drive C, a striped set. No issue with Drive C. Data was on Drive D, a RAID Set with 12 drives, 2 in a failed state. So no data there. (Otherwise I wouldn’t be chasing this.) Also, made a copy of this file, just in case my Ghost copy gets scrubbed. (It’s on a “server” in tulsa, and I’m in Texas)
I have 2 Dlist Entries both marked “Temporary” with a size of “-1”.
Adding more unknowns… Well, I guess you can either pick one at a time for testing, or hit both at once.
You need to have a backup saving to the for-experiments copy of your original backup files, and I guess Database path is at the DB you’re in. If so, edit and save to the for-experiments copy. Fix path if needed.
Try the Repair button on Database screen and see if a dlist or two gets put up in the experimental area.
There are some other possible things to try. This is a bit involved, but might be best given general mess.
Thanks. This worked. Even though the backupset apparently never completed, It at least allowed me to get a list of what was backed up, and am able to restore some things.
Summary:
Made copy of SQLLite database w/ Random generated name from C:\Users\Username\AppData\Local\Duplicati.
Opened in SQLLite Browser.
3.Looked for Dlist entries in RemoteVolume Table, and Changed setting from Temporary to Uploaded. Clicked Write Changes. Exited SQLLite browser.
Restarted Duplicati.
Went to Advanced, Database on the BackupSet. Changed the File Location to match the copy I am working with. Clicked Save.
Restarted Duplicati again. (As it threw an access denied error initally, probably from a stale lock from SQLLiteBrowser.)
Went to restore, it built the file list. I am now restoring everything from the Data Drive that is available.
Thanks all for the help, may have just saved my Bacon!
If you didn’t Repair to make a dlist file, then the DB edits might not have been needed, but it was fast.
Possibly your original restore attempts were “Direct restore from backup files” which would need dlist. Duplicati.CommandLine.RecoveryTool.exe can handle corrupted backups better, but still needs a dlist.
Getting the DB in that matched the backup destination might have been enough. I hope it finishes well.
The dlist is made at the end of a backup, so early fail doesn’t upload a new one (or here, the first one).
This (or a matching database) say what’s in the backup and how to put the blocks together for restore.
Of the other items on the issues list, the only one that rings a bell is that the GUI for filters is a bit off in current Beta (fixed in Canary), so use the three-dot menu and Filters to see if GUI did what you hoped. The TEST-FILTERS command, e.g. run from Commandline, can test filters but filters can be complex.