I have been trying to diagnose an occasional error being returned by TestHandler in a backup job, and I am resorting to the use of the command line utility, in particular the “affected” command. At first I was getting an error related to a non-existing database file. I have since discovered that the problem is that (as referred in this related thread) Duplicati tries to “automagically” find out which local database correspondes to the storage URL. After specifying the correct local DB as an argument, the command worked as expected.
I suspect in my case this was problematic because over time I have created different backup jobs all pointing to the same storage URL, some of which were deleted and some of which I still use. This leads me to a question and what I think is a way to address multiple local DBs:
Why would the current logic map a local DB which has since been deleted?
When multiple DBs are detected by the affected command, a warning on this fact could help the user better diagnose the behaviour of the command. Even better would be an explicit question on which local-db to select - perhaps also supplementing the local-db filename with the name of the corresponding backup job, if it exists.
Are there potential pitfalls on this approach? (One that comes to mind is confusion created by local databases that are temporarily created for various purposes).
The phrasing of the first two paragraphs sounds like you are creating (and deleting) jobs in Duplicati GUI, however the easiest way to run affected would be in GUI Commandline where DB mismatching is rare.
Export As Command-line is available if you really have to use raw OS facilities, and dbpath is specifically exported, and so the automagic matching based on destination URL never goes into use to confuse you.
DB mapping and existence are somewhat separated. For example. GUI maps before doing inital backup. Possibly deleting a CommandLine destination and DB (if you can find DB) would recreate on backup too.
The GUI and CommandLine run independently by design. If you deleted a DB in the GUI, only GUI knows.
If I guessed wrong and you somehow found a DB and deleted it at the command prompt, nothing knows.
That’s an extremely common scenario in the CommandLine. You’re supposed to give the destination URL which cuts it down to the right DB for that backup. The affected command is “supposed” to not be much used, and the usage you’re talking about sounds like asking “Is the file used by any of my backups? How?”
Such a query seems like it could occasionally be helpful, but I’m not sure there’s a huge case to prioritize.
Things are not “supposed” to get so highly puzzling. A job has a destination, backup files, and a database. Staying in the GUI keeps track of all that. Staying in CommandLine does too. Mixing these can use Export.
I am actually running the “affected” command through the command line. I did not know that you could even run it from the GUI. Perhaps I gave that impression when I talked about “different backup jobs all pointing to the same storage URL, some of which were deleted and some of which I still use”? In that sentence, I was talking about actual backup jobs (“backup” command and not “affected”) that I have created over the last 4 years (due to various reasons), some of which I have since deleted. I suppose the non-existing database that the affected command identifies belongs to one of those old deleted jobs, otherwise the alternative is that it is just making one up? Here is the output of the “affected” command without explicit --db-path:
.\Duplicati.CommandLine.exe affected "onedrivev2://<censored URL>" duplicati-b30fdb28392d949baa72086fe28cedb25.dblock.zip.aes
ErrorID: DatabaseDoesNotExist
Database file does not exist: C:\Users\<censored username>\AppData\Local\Duplicati\HSSLMPBJXW.sqlite
The database file referred by the output does not exist and does not belong to any backup job I have currently configured in the GUI.
What I was trying to answer with the affected command (refer to the citation of the command I was using above) was: in the backup job pointing to the OneDrive URL “onedrivev2://”, what files are related to the block duplicati-b30fdb28392d949baa72086fe28cedb25.dblock.zip.aes? Unfortunately, a run of the command just with the backend URL (and no --db-path) errors out and identifies a “database file does not exist”. The database referred, entirely “auto-discovered” by the command, is indeed a non-existing database file. I suppose it may be from an old backup job that may have had the same backend URL, but other more remote possibilities exist (eg. the command just making a database filename up, does not seem probable though).
Deletion is what gave me the impression you used GUI backup jobs. CommandLine has no stored jobs.
Seeing new destination URL seems to assign DB in advance of its use. Note I said the GUI did similarly.
For CommandLine you can look inside the dbconfig.json file in the folder with the DB. It tracks all this.
That’s because either you give it the --dbpath from the GUI job (which it doesn’t know – it’s independent),
or it assigns DB path, then finds there’s nothing there. Ordinarily you run a backup before asking about it.
Export As Command-line gives you a ready-to-run backup command. If you want some other command then the line must be modified to give desired command and format. Same is true for GUI Commandline.
Syntax for affected is simple, so you could also get the Local database path from Database screen. Connecting CommandLine to an existing job requires at least giving it the --dbpath that the job was using.
Thank you for the detailed write-up. So if I understand correctly:
Duplicati separates the GUI and command-line world in such a way that a backup job set up in the GUI will not be autodiscovered by the “affected” command running on the command-line.
When the “affected” command is run without --db-path, if it does not successfully autodiscover a database, it will assign a new db-path, and because this new db-path does not exist, it will exit with the error “DatabaseDoesNotExist”, referring a database file that indeed does not exist (because it was just assigned, and not actually created, by the very same “affected” command).
Is this understanding correct?
If yes, couldn’t this be improved by the “affected” command not assigning a new db-path when it does not discover one? It would not address the GUI/command line separation (which depending on the perspective might actually be a feature), but it would prevent an obscure error.
Regarding the way you described to obtain the --db-path from the GUI and specifying it in the command line, it absolutely does work, and it was how I actually ran the command after understanding that the “affected” command was not being successful autodiscovering the dabatase path.