Yesterday had what appears to be a corruption of the database in my docker container running Duplicati Beta. Duplicati will now no longer start up.
I tried copying one of the ‘backup’ databases over the config database, but now when I try to start up I get:
duplicati | A serious error occurred in Duplicati: System.Exception: Failed to create, open or upgrade the database.
duplicati | Error message:
duplicati | The database has version 7 but the largest supported version is 6.
duplicati | This is likely caused by upgrading to a newer version and then downgrading.
duplicati | If this is the case, there is likely a backup file of the previous database version in the folder /config.
duplicati | at Duplicati.Server.Program.GetDatabaseConnection (System.Collections.Generic.Dictionary`2[TKey,TValue] commandlineOptions) [0x00229] in :0
duplicati | at Duplicati.Server.Program.RealMain (System.String _args) [0x002d0] in :0
How can I use one of these backup databases to get Duplicati up and running again?
Did you change your Duplicati version? If so what version were you running and what are you now running?
If you were running Duplicati version 220.127.116.11-18.104.22.168 and decided to install 22.214.171.124, that was actually a downgrade. 126.96.36.199 is special release with an unfortunate version number.
Ah, it looks like it is indeed running 188.8.131.52. Any way to use these backup databases then?
Given that 184.108.40.206 was a ‘downgrade’, but managed to use the database from the previous version, can I just change the version number in the database using sqlite3?
Don’t hand-edit the version in the database…you’d be asking for trouble!
What version were you running before? I’d probably just go back to that version. Or since it was presumably a canary release just use the latest canary, 220.127.116.11. it is pretty solid in my experience.
I don’t know what version it was. I’ll ask the linuxserver.io people. Pretty sure it won’t have been a Canary though.
The beta channel would not have experienced this. Perhaps you were running experimental? 18.104.22.168 was the last such release and if you went to 22.214.171.124 later it would have caused your issue.
BTW Duplicati has its own official docker releases… You don’t need to use a third party like linuxserver.io.
Almost certain that container was only ever on Beta.
Didn’t know about the official releases, will have a look. I like linuxserver’s approach in general though.
Thanks for the pointer.
The container has only ever been on 126.96.36.199 since I started it I believe. The previous beta release by them was over 2 months ago, and I don’t think the container has been up that long.
Can’t understand how these backup databases have v7 in them then?
Would going back to v188.8.131.52 help?
Ok, I’ve just reset the container to empty, and started it up.
Created a single backup and run it.
I get this:
andy@xcp-docker:~/docker/duplicati$ su -c “sqlite3 config/backup\ 20190912111623.sqlite ‘select * from Version;’”
andy@xcp-docker:~/docker/duplicati$ su -c “sqlite3 config/Duplicati-server.sqlite ‘select * from Version;’”
So as you can see, the main database has version 6, but the backup database has version 7.
I’ve just double checked, and it’s the same on my non-dockerised version, which is still running 184.108.40.206_beta_2018-11-28.
root@xcp-dl:~/.config/Duplicati# sqlite3 backup\ 20190901022923.sqlite ‘select * from Version;’
root@xcp-dl:~/.config/Duplicati# sqlite3 Duplicati-server.sqlite ‘select * from Version;’
Something amiss here surely?
Wait a minute, I think you may be comparing two different types of databases.
The main Duplicati engine database is
Duplicati-server.sqlite and even the latest Canary release has that at version 6.
The other type of database is the job-specific database, and its name is a string of 10 random letters. (Although there was a bug in some versions of Duplicati that caused this 10 letter name to be converted to a string of ASCII values for those letters.)
The job-specific databases may have a different version than the main engine database. This is because the schema for these two database types may be changed at different points during development for different reasons, since the databases have different uses. On my canary 220.127.116.11 system the job-specific databases are at version 9.
Also, I notice in your example that you are checking a database that has the
backup prefix in its name. This is a one-time copy of a database Duplicati makes when it needs to upgrade the schema of a database. After that copy is made the databsae is no longer ever used. They are not the active, job-specific databases.
Unfortunately the older versions of Duplicati don’t include the original part of the filename when creating these one-time database backups, so it can be hard to tell if it was originally a job-specific database (and for which job) or the main Duplicati engine database.
I’m still not sure exactly what happened in your case, but I think you must have upgraded beyond the beta channel at some point. I have seen other docker users say they have upgraded Duplicati within the container, so maybe that is what happened in your case? (Personally I would not do this but instead just delete the container and download a fresh image when a new version is released…)
Are you willing to test a newer experimental or canary version with your databases?
I assumed the ‘backup’ databases were backups of the main Duplicati configuration database.
I’ve since worked out why the container won’t start. It’s actually something to do with specifying the path to the file to use as the SSL key. I can create a clean container passing this parameter. Do nothing, then stop and restart it, and it’ll then fail to start.
So at least I’m back up and running, and I think you’ve explained the confusion over the backup databases.
Rest assured that I’m now backing up duplicati’s databases via another means (borg if it’s of interest) to avoid this kind of situation moving forward.
Thanks for your help @drwtsn32, you’ve at least cleared up a few misconceptions of mine! There’s no chance the software either in this container or the non-container version has ever been on a recent Canary build. I moved to the beta channel a year or two ago on the non-container version, and the container has only been running for two or three weeks, and has always been on the beta channel.
It can be either - fortunately newer versions of Duplicati keep part of the original filename in the backup file so that they are easier to identify.
Then I have no idea how this happened. Maybe someone else has a different theory.
Glad you are up and running!
Basically the same theory, but because server database has not reached version 7 (still at 6), version 7 could be a job database. If you stuck to beta releases, it couldn’t be 18.104.22.168 or 22.214.171.124, but 126.96.36.199 used a version 7. Easiest way to tell is to view structure with tool such as sqlite3.exe or sqlitebrowser (DB Browser for SQLite), e.g. if there’s a table named Backup, that used to be a Duplicati-server.sqlite. A Block table was a backup DB.
EDIT: Fixed 188.8.131.52 info, and here’s how it looks: