I updated the post to include using the button for making changes. Hopefully it works that way for people who aren’t logged in as me.
I tried downgrading from 220.127.116.11 Canary down to 18.104.22.168 Beta. Then every time Duplicati server started, it would immediately crash with no error message. When run as a service it appears to be working, but the actual server isn’t running.
I tried running via command line and finally saw the error below. It would be nice if this showed up with the GUI or service. I deleted the file C:\Users\User\AppData\Local\Duplicati\Duplicati-server.sqlite and it now starts correctly. Note that this is a new PC I’m testing on with no existing backups so deleting database files isn’t an issue (in case that matters).
C:\Program Files\Duplicati 2>duplicati.server
A serious error occurred in Duplicati: System.Exception: Failed to create, open or upgrade the database.
The database has version 5 but the largest supported version is 4.
This is likely caused by upgrading to a newer version and then downgrading.
If this is the case, there is likely a backup file of the previous database version in the folder C:\Users\User\AppData\Local\Duplicati.
_ at Duplicati.Server.Program.GetDatabaseConnection(Dictionary`2 commandlineOptions)_
_ at Duplicati.Server.Program.RealMain(String args)
Yes, it would be nice but the errors are logged into the database so if Duplicati can’t get to the database then there is nowhere to log the error.
I’m not sure why you got the error in the first place as I didn’t think there was a database version change between 22.214.171.124 and 126.96.36.199, but thanks posting it. I’ll update the main post to reflect that.
https://github.com/duplicati/duplicati/tree/v188.8.131.52-184.108.40.206_canary_2018-04-02/Duplicati/Server/Database/Database%20schema is where the server database went to version 5, and I see that we’re now up to 6…
Although it’s not a great solution, sometimes these seemingly silent crashes leave a Duplicati-crashlog.txt although I’m not sure where they go. I’ll show the start of one that’s sitting in my Duplicati “updates” folder:
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Exception: A serious error occurred in Duplicati: System.Exception: Failed to create, open or upgrade the database. Error message: The database has version 5 but the largest supported version is 4. This is likely caused by upgrading to a newer version and then downgrading. If this is the case, there is likely a backup file of the previous database version in the folder C:\ProgramData\Duplicati.
This shows how to get from 5 to 4, however downgrades unfortunately have to be tailored to their upgrades.
Great info - and thanks for the link. I’ve updated the main post with all the database upgrade versions to date.
And more could be said, but there might be a shift in emphasis, maybe aimed primarily at those using Beta.
There’s now a recent safety recommendation to consider about how to do downgrades. While Canary users should be aware that Canary is a risk (even risky downgrades may help them), what should other users do? Canary moves in smaller quicker steps, so downgrade is easier, whereas a Beta-to-Beta downgrade is a lot.
Having this article provide steps for the recommendation below would be an approach. I’ve never been that comfortable with people editing databases manually, but there’s no programmed method - and no test suite. One reason for discomfort is that while structure changes are obvious, changed structure usages are not… Someday there might also be a big-time database redesign for performance, reliability, logging, whatever…
So here’s the conversation at Any progress in new beta? but I don’t know if “bulletproof” changed the reply:
Thanks - steps to use the automatically created database backup have been added to the main post.
@kenkendk, are these backups ever cleaned out? If not, we might want to let people know that every Duplicati update creates another copy of their database. I’ve heard of some people with 1-2G databases and that could add up over time…
We should likely speak of “database” in the plural, which also begs the question of how to tell what was what, even to the extent of distinguishing former Duplicati-server.sqlite from former database for some backup job.
How to test which database belongs to which job? gets into this, but the solutions discussed are rather crude even before getting into Windows database encryption (I wonder if backups follow --unencrypted-database?).
Nuts - you’re right, it’s unclear based on name alone whether the backup is for
Could you please update this with the latest database versions?
@AimoE, thanks for pointing out that database b versions have changed. I haven’t had time to keep up with the latest versions but I’ll try to take a look in a few days.
Note that the first post should be a wiki so if you already know the release and database numbers you can update it yourself by using the Edit button at the bottom right of the post.
But I don’t know the versions, how could I? That is exactly why I hope you would update the info.
Before you publish the first official version of Duplicati, you need to learn to provide users the basic compatibility information, such as db version changes, with the releases (plus link to full list of changes).
Sorry, I thought you had requested an updated list because you knew a database version had changed.
I figured you had learned it in one of the same way I do:
- review pull requests
- see it mentioned in a release announcement
- experience or hear of a database specific issue with a downgrade.
If you find something about Duplicati lacking please feel free to join in and help us improve it! If it’s not happening now it’s likely because we either don’t know how or don’t have the time to do it.
You’ll find the context for my request here: Release: 220.127.116.11 (beta) 2019-07-14
I used the information from that post to update the wiki. Feel free to change it if you think it unclear or incorrect.
But you didn’t add the information I was asking for. The error message I got from the latest beta mentioned database versions 8 and 9. Your posting stops at version 7. Which Duplicati version came with db version 8? Which came with 9?
My guess (on a thread you were on) is below. Would you consider donating some hours to explore this? Basically, it’s a whole lot of clicking plus record-keeping. Retroactive testing is probably too much to ask. Previous note here also points out that there is no downgrade test suite, so precision data loses value…
Error accessing database version 9 plus a note that 18.104.22.168 does 8, and I’m not sure I want to go further, however I suspect the table at top could use confirmation, and there are also two different DBs involved.
I’m thinking v22.214.171.124-126.96.36.199_canary_2019-01-29 through (so far) v188.8.131.52-184.108.40.206_canary_2019-06-30 however if you’re not testing bleeding-edge canary releases, v220.127.116.11-18.104.22.168_experimental_2019-06-28 might be what you had.
Basically, to be the most helpful, one would say where a DB got added on each of the release channels. Databases for the server DB and the backup DB go independently, but either one can hurt a downgrade.
That did occur to me too at some point. Unfortunately, despite its lengthy wording, the error message I received does not specify which db is at question. Maybe I should place a bug report on that in your long queue of bugs.
So the only way to find out what I need to know is try downgrading and see if it succeeds. I am principally interested to know if I have a chance to downgrade from latest canary to latest experimental and then maybe some day get it upgraded to beta.
Can you promise that if any one of the two dbs isn’t compatible after a downgrade, I will get an error message at first attempt to run a job, not down the line later?
Not exactly. This whole topic is sort of in violation of the it-hasn’t really-been-tested caveat, but at least the click-source-tree plan (don’t even need to sign into anything) can tell you when DB version WILL complain. As a side note, I can’t recall any downgrades that didn’t work due to something other than the DB versions.
Based on what I wrote (assuming I wrote/re-read it right), latest experimental and latest canary should play fine on DB version. Presumably move forward to next beta would also work. Downgrade to 22.214.171.124 will not.
At the moment, latest
Canary is v126.96.36.199-188.8.131.52_canary_2019-06-30
Experimental v184.108.40.206-220.127.116.11_experimental_2019-06-28, and IIRC the actual code is basically rebuild of v18.104.22.168-22.214.171.124_canary_2019-06-25 (sometimes an Experimental is custom-blended, but this was not).
Because downgrade from latest canary to latest experimental is only a small step, possibly your reason is getting off the Canary upgrade treadmill (which by design is bleeding-edge). If so, you can instead change channel to Experimental in Settings, overriding the default of Canary channel you have if you install Canary.
Getting information about your setup (specifically System info tab) might help before you change things, especially since you gave a thought in some topic that you got an update in violation of the channel plan. Info from these two screens I mention (Settings, and System info) might give a clue on what happened.
I don’t know the code that well, but I believe the check is at open time. The server DB opens right away to even show you your backup jobs screen. Some specific backup DB is probably not opened until required. There is a backup taken (as mentioned earlier), and one change in recent versions is now the backup file name includes the name of the file it came from (as opposed to just a date, leaving one a difficult guess).
So while I’m not making a firm promise, Duplicati doesn’t AFAIK handle multiple DB versions at once. Its solution is to upgrade (as required) on first use (unless it does everything in sight – look at your results).
So I seriously doubt a late DB version complaint is possible.
What’s definitely possible is that the backup of the old DB taken at version update time will grow obsolete rapidly as new backups occur. Trying to revert to it as part of a downgrade will mean losing new versions. Possibly this is what you were getting at. There’s a small time window on a given backup to go back, and possibly testing a less critical backup first when you do a major version jump would be the safer method.
I didn’t get update, I got notification about the new beta, and even though I have already dismissed the notification, I got it again, possibly after rebooting. I already updated my release channel setting, but that didn’t help with the notification, I still got it. This insistence may trick some users, I warn you.
The “on first use” part here is what worries me, so let me rephrase my question: does the “first use” of each db happen before doing anything else or can it happen when Duplicati has already done something for the job run at hands?
If I cannot trust that all db version checks precede all other steps, then I will have to try the code excavation route, but I don’t particularly like being pushed into it.