Duplicati Error : It says there are 36 files missing from remote storage, but :
1> First, I don’t have remote storage, only local, so it’s reporting an error that can’t even exist.
2> If I try to check the failed files, it doesn’t display. As I don’t have remote storage and such files don’t exist, it seems tautological that it doesn’t display them. But this takes away the confidence that Duplicati will restore lost files, when in fact lost files do exist – It’s just the old situation where a false positive is worse than the problem itself. When I say it doesn’t show them, I mean the interface buttons don’t even respond, it’s like clicking on a stuck page. I don’t even know if this lack of feedback is normal in Duplicati.
I get a dim feeling from your post that you are backing up to a removable disk (USB maybe ?) and that you are trying to use a new disk on the same backup job is it the case ?
The description of your problem is far from clear in my opinion, I have no idea on what can be the ‘buttons’ you are talking about.
Well, returning to Gpatel-san, who was kind enough to take an interest in the problems of this newbie who writes to you: No, the drive is not removable; it’s a 2Tb HD with 2 partitions, one of 500Gb, it’s a database (it’s more like a dump folder, I just throw the files there, inside several subfolders after sorting the material), and the other of 1, 5Tb is the backup database of this first partition and the Home folder, which is on a separate SSD (the system is Linux, with separate System, Home, and Backup partitions).
As for the aforementioned buttons, when an error message appears, the page offers 3 buttons in the footer: “OK” (meaning ignore errors, I suppose), “Show”, and “Restore”. “OK” is the only one that does something, which is “nothing” – “OK” just clears the error prompts, and the system tray icon changes from red to colorless; hence I assumed that “OK” means to ignore errors. The next two buttons, “Show”, and “Restore”, neither show nor restore the failed files, so I have serious doubts about what they mean. System slowdown is not, since this is a very powerful machine (i5 5th generation, with 16 Gb of memory, and the system and Home partitions are on SSD separate from the HD). If with so much power a button doesn’t respond, after 8 minutes of waiting, it’s because for some reason the button doesn’t work at all…
I humbly advance that although I know how to build and deconstruct a Linux system; I’m new to Backup tools. Until recently, my “backup technique” consisted of copying all my files by hand on another partition, well, pretty redneck…
Well let’s try to resume. You seem to have:
- a SSD drive
- a classic hard disk with 2 partitions.
You are doing a backup (partial ?) of the SSD drive to the second partition.
You are doing an archive of the first hard disk partition to the second partition.
When Duplicati is talking about the ‘remote’ device, it’s probably really your hard disk second partition. That’s just the way it’s said, because most Duplicati users are backing up to an Internet device (cloud).
Aside:
I say an archive because a backup is supposed to bring more reliability to your data by having redundant hardware. If you copy data from a hard disk partition to another, you have no better reliability. If your hard disk fails you lose everything, so it’s not really a backup.
Anyway, my guess is that you are trying to backup but you either moved the backup files to another place or changed to backup destination in the backup configuration. In this case the user interface gives 3 options:
- repair: try to fix the problem, but in this case if will always fail, a simple repair is not enough.
- show: means: show the log window
- dismiss: don’t do anything, just close the warning popup
My guess is your UI is not translated correctly, what is the language of your system ? It’s possible that the UI don’t work correctly on your system either, I have no experience of backinp up a Linux system with Duplicati, I use it only with Windows.
“remote storage” = destination, be it local(internal), removable(USB) or remote storage(internet). “You have missing files in your destination.” could be a better way to describe the error but that’s all it means.
As for having missing files in the destination there are a few ways this can occur, most often after an incomplete (interrupted) backup. As for how to fix it that depends on how invested you are in that backup set.
To be super safe I’d make a copy of the files currently in the remote storage along with a copy of the database for that backup job, then try running a Repair (Open the backup job properties, click on Database then Repair). If that doesn’t fix it, it may at least provide a different error.
If that doesn’t work you may need to use the command line tools to clean up the files --list-broken, --purge-broken or use the Command line Recovery Tool.
If you’re not concerned about the contents of the backup and just want to get it running a fresh then delete the whole thing and recreate it. Delete the backup job from the GUI and remove all the .dlist, .dindex and dblock files from the destination then recreate it.
Side note: Files can be restored without the database so long as you have the .dlist, .dblock and .dindex files. You can give the Disaster Recovery part of the manual a read for a more info. The manual section on Recovering files from a lost installation is also quite helpful.
Hope this helps.
Can you paste a screenshot? English buttons should look like below:
I’m not a translation expert, but your OK worked like Dismiss should:
Error popup image above was dummied up by hiding a dindex file because sometimes an interrupted backup at a vulnerable point (specifically, a compact which is cleaning up wasted space) can get this.
The Repair button will clean it up. Was this backup running successfully for awhile before this issue?
About → Show log → Live → Warning in a different tab should give details on which files are missing.
You can try running the backup again or you can use the Verify button. Preferably it’s just dindex files.
Also preferably it’s the known issue with them that the Repair button can handle. Needs more testing.
EDIT:
pt_BR language code is Portuguese (Brazil). pt is Portuguese. I’m not familiar with what’s different.
Possible translations of Dismiss might be Dispensar or Descartar. Translation is done by Transifex.
Thanks to everyone for the clarification. The buttons are the same as shown in ts678’s comment, so… First, yes, the language is Portuguese-Br (Brazilian Portuguese); second, “dismiss” is the “OK” that appears on my system. The “Show” button, “Show” now works, it displays the list of Previous Backups. The problem is the “Repair” button, which misleadingly suggests some sort of automatic restore, but Jimbo Jones clarified that it’s just a sort of clean button, although I still don’t quite understand what “clean” actually is – I’m new to these backup spells, practically a hick; so things like “dindex” files sound like pre-Mesopotamian Aramaic to me, but from the name, I’m assuming it’s some sort of index file… If I’m not guessing wrong – and I’m guessing too much already, I think – the warnings refer to missing files that I deleted from the original. Correct me if I’m wrong. And as for Jimbo’s question, the system had been doing backups normally for days (I started just 20 days ago), and in that period, I also deleted several files, without any warnings. So of course, popping up 36 warnings at once 3 days ago caught me totally unprepared. I tried the repair button again to see what happens now. I’ve looked at the log, but it only refers to the backup compressed files, with no clue as to what the missing files might be. OR, I would be misunderstanding, and the warnings are about the first 3 backups I did as a test, and just as a test, and then discarded. They appear in my list of backups in yellow, unlike later backups which appear in green. Is this the expected behavior? How do I totally eliminate these yellow ones from the list? I would like to see if this would eliminate the warnings entirely, if things are still not resolved. Thanks for the help guys!!
yes the translation is wrong. You can help translate it correctly for your language by using https://www.transifex.com/duplicati/duplicati/ (no programming skills required)
do not delete files from the backup location manually You should always delete backup data from the User Interface as files are tracked in the local database and doing so will create a mismatch between what the backup software knows about the ‘remote’ (backup data) and the reality. Best is to limit the space taken by your backup by changing retention policy as Duplicati will take care of removing old data by itself.
Now to fix this you need to use command line I think. You can actually run the ‘command line’ from the User Interface, I’d try to use purge-broken-files (you have to clear the box for the paths and remove any exclusion for this to work)
As a general advice, could you try to make your posts more readable by separating the different points that you want to say into paragraphs ? It’s a small effort to hit ‘enter’ from time to time, but it comes some way to make life easier for people trying to help you.
1 Like
It’s not clear where you deleted files (source should be fine, destination not), and which files you want information on. As you say, it tells you exact name of the missing destination files, if I understood you.
What missing destination file names are you getting? That would help understand the best next steps.
While the message is about destination files, you can translate to impact on source in list-broken-files before actually doing the purge-broken-files, but this works best if your missing filenames say dblock.
Thanks everyone again. For my part, I’d like to know how to delete the index files and any leftovers from the first 3 attempts I made – May 12th, 11th, and 9th; and that as pointed out, I made the mistake of manually deleting ( because I didn’t find in the interface a simple way to delete them. In the end, I lost patience, entered the partition with root permissions < superuser > and manually deleted the files ). As I understand it, I’m going to need the command line, but tools like the “Command line Recovery Tool” are out of the question, as it’s an .exe tool, and this one is a Linux system. Maybe the tool even works via Wine ( which allows you to run .exe applications on Linux ) But Wine is notoriously difficult to operate and often fails. It is for banal cases that use dependencies that are common between Linux and Windows (like ffmpeg), but when asking for specific dependencies or that are only available for Windows, it fails completely. In practice it’s good just to use Winrar on Linux, and some well-known games. Summing up the nenia: I’ll have to use the cold command line, without anesthesia. For that I need precise guidance. On Linux we use the command line a lot, it is familiar to us, and among veterans there is even a certain “rancid” against GUI interfaces, and sometimes it is even necessary to get on the foot of certain command line fanatic people who do preaching against GUI interfaces, explaining to them that GUI is not for lazy, ignorant, beginners ( which is all they think of Windows users < “Ruinwindows”, as we say > ), but that GUIs are for what user doesn’t get the syntax and attributes wrong, or have to memorize the syntax and attributes of hundreds of commands, some of which he will only have to use once or twice in his life.
So, having said that, what would be the correct syntax to use ? Where is a tutorial on commands and syntax for basic cases on the command line ?
Not all .exe files are Windows code. Duplicati’s .exe is a portable .NET Assembly. See below.
Assemblies in .NET (Microsoft)
Assemblies are implemented as .exe or .dll files.
Mono Basics shows how to use mono hello.exe to run the program built there. Ignore details.
Duplicati is already built. You either run it on Linux under mono, or some Linux just “know” how.
> cat /etc/issue
Linux Mint 19.3 Tricia \n \l
> /usr/lib/duplicati/Duplicati.CommandLine.exe
See duplicati.commandline.exe help <topic> for more information.
...
On Windows, Duplicati’s portable code runs under .NET Framework which Windows has installed.
On Linux, you do have to make sure that mono --version has at least 5.10. Old Linux can be old.
Using Duplicati from the Command Line in the manual talks about the primary tool, including mono.
On the right side of the page are the Duplicati commands that Duplicati.CommandLine.exe can do.
Putting it all together from scratch is easy to get wrong. Safer to use GUI Export As Command-line.
Other Command Line Utilities describes the specialty commands, including those that serve the UI.
Look at the file name. A dindex file’s name has dindex in it, and dblock and dlist are similarly named.
You don’t even need to look. Just copy and paste the names of whatever files are missing in live log.
So you now have 36 files missing, and those are all manual deletions, and you don’t recall the type?
Still unclear which files or why. Would you consider deleting them all and running backup fresh again?
Database delete should be done along with the destination file delete. DB must match the destination.
Yet another viewpoint:
Remote means the backup location. Online or offline it doesn’t matter.
I’d say just delete the Duplicati backup files (all of them) and then in Duplicati click the down arrow on the task > database > delete. And run a whole new backup. Problem solved. You can then move onto other things while it runs.
If you’re one that thinks its great to do a 1TB or greater backup then cut that monster into smaller chunk’d tasks and then run. Less fuss in the future.
However, if you need to restore from the current backup then you would have to deal with trying to get it working and that’s 50/50 at best.