Downgrading / reverting to a lower version

Yes, it would be nice but the errors are logged into the database so if Duplicati can’t get to the database then there is nowhere to log the error. :frowning:

I’m not sure why you got the error in the first place as I didn’t think there was a database version change between 2.0.3.3 and 2.0.3.6, but thanks posting it. I’ll update the main post to reflect that.

https://github.com/duplicati/duplicati/tree/v2.0.3.4-2.0.3.4_canary_2018-04-02/Duplicati/Server/Database/Database%20schema is where the server database went to version 5, and I see that we’re now up to 6…

Although it’s not a great solution, sometimes these seemingly silent crashes leave a Duplicati-crashlog.txt although I’m not sure where they go. I’ll show the start of one that’s sitting in my Duplicati “updates” folder:

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Exception: A serious error occurred in Duplicati: System.Exception: Failed to create, open or upgrade the database.
Error message: 
The database has version 5 but the largest supported version is 4.

This is likely caused by upgrading to a newer version and then downgrading.
If this is the case, there is likely a backup file of the previous database version in the folder C:\ProgramData\Duplicati.

This shows how to get from 5 to 4, however downgrades unfortunately have to be tailored to their upgrades.

1 Like

Great info - and thanks for the link. I’ve updated the main post with all the database upgrade versions to date. :slight_smile:

And more could be said, but there might be a shift in emphasis, maybe aimed primarily at those using Beta.

There’s now a recent safety recommendation to consider about how to do downgrades. While Canary users should be aware that Canary is a risk (even risky downgrades may help them), what should other users do? Canary moves in smaller quicker steps, so downgrade is easier, whereas a Beta-to-Beta downgrade is a lot.

Having this article provide steps for the recommendation below would be an approach. I’ve never been that comfortable with people editing databases manually, but there’s no programmed method - and no test suite. One reason for discomfort is that while structure changes are obvious, changed structure usages are not… Someday there might also be a big-time database redesign for performance, reliability, logging, whatever…

So here’s the conversation at Any progress in new beta? but I don’t know if “bulletproof” changed the reply:

Thanks - steps to use the automatically created database backup have been added to the main post. :slight_smile:

@kenkendk, are these backups ever cleaned out? If not, we might want to let people know that every Duplicati update creates another copy of their database. I’ve heard of some people with 1-2G databases and that could add up over time…

We should likely speak of “database” in the plural, which also begs the question of how to tell what was what, even to the extent of distinguishing former Duplicati-server.sqlite from former database for some backup job.

How to test which database belongs to which job? gets into this, but the solutions discussed are rather crude even before getting into Windows database encryption (I wonder if backups follow --unencrypted-database?).

Nuts - you’re right, it’s unclear based on name alone whether the backup is for Duplicati-server.sqlite or [job].sqlite.

Could you please update this with the latest database versions?

@AimoE, thanks for pointing out that database b versions have changed. I haven’t had time to keep up with the latest versions but I’ll try to take a look in a few days.

Note that the first post should be a wiki so if you already know the release and database numbers you can update it yourself by using the Edit button at the bottom right of the post. :slight_smile:

But I don’t know the versions, how could I? That is exactly why I hope you would update the info.

Before you publish the first official version of Duplicati, you need to learn to provide users the basic compatibility information, such as db version changes, with the releases (plus link to full list of changes).

Sorry, I thought you had requested an updated list because you knew a database version had changed.

I figured you had learned it in one of the same way I do:

  • review pull requests
  • see it mentioned in a release announcement
  • experience or hear of a database specific issue with a downgrade.

If you find something about Duplicati lacking please feel free to join in and help us improve it! If it’s not happening now it’s likely because we either don’t know how or don’t have the time to do it. :slight_smile:

You’ll find the context for my request here: Release: 2.0.4.23 (beta) 2019-07-14

I used the information from that post to update the wiki. Feel free to change it if you think it unclear or incorrect.

But you didn’t add the information I was asking for. The error message I got from the latest beta mentioned database versions 8 and 9. Your posting stops at version 7. Which Duplicati version came with db version 8? Which came with 9?

My guess (on a thread you were on) is below. Would you consider donating some hours to explore this? Basically, it’s a whole lot of clicking plus record-keeping. Retroactive testing is probably too much to ask. Previous note here also points out that there is no downgrade test suite, so precision data loses value…

Error accessing database version 9 plus a note that 2.0.4.5 does 8, and I’m not sure I want to go further, however I suspect the table at top could use confirmation, and there are also two different DBs involved.

I’m thinking v2.0.4.13-2.0.4.13_canary_2019-01-29 through (so far) v2.0.4.22-2.0.4.22_canary_2019-06-30 however if you’re not testing bleeding-edge canary releases, v2.0.4.21-2.0.4.21_experimental_2019-06-28 might be what you had.

Basically, to be the most helpful, one would say where a DB got added on each of the release channels. Databases for the server DB and the backup DB go independently, but either one can hurt a downgrade.

https://github.com/duplicati/duplicati/tree/master/Duplicati/Server/Database

https://github.com/duplicati/duplicati/tree/master/Duplicati/Library/Main/Database

That did occur to me too at some point. Unfortunately, despite its lengthy wording, the error message I received does not specify which db is at question. Maybe I should place a bug report on that in your long queue of bugs.

So the only way to find out what I need to know is try downgrading and see if it succeeds. I am principally interested to know if I have a chance to downgrade from latest canary to latest experimental and then maybe some day get it upgraded to beta.

Can you promise that if any one of the two dbs isn’t compatible after a downgrade, I will get an error message at first attempt to run a job, not down the line later?

Not exactly. This whole topic is sort of in violation of the it-hasn’t really-been-tested caveat, but at least the click-source-tree plan (don’t even need to sign into anything) can tell you when DB version WILL complain. As a side note, I can’t recall any downgrades that didn’t work due to something other than the DB versions.

Based on what I wrote (assuming I wrote/re-read it right), latest experimental and latest canary should play fine on DB version. Presumably move forward to next beta would also work. Downgrade to 2.0.4.5 will not.

At the moment, latest
Canary is v2.0.4.22-2.0.4.22_canary_2019-06-30
Experimental v2.0.4.21-2.0.4.21_experimental_2019-06-28, and IIRC the actual code is basically rebuild of v2.0.4.20-2.0.4.20_canary_2019-06-25 (sometimes an Experimental is custom-blended, but this was not).

Because downgrade from latest canary to latest experimental is only a small step, possibly your reason is getting off the Canary upgrade treadmill (which by design is bleeding-edge). If so, you can instead change channel to Experimental in Settings, overriding the default of Canary channel you have if you install Canary.

Getting information about your setup (specifically System info tab) might help before you change things, especially since you gave a thought in some topic that you got an update in violation of the channel plan. Info from these two screens I mention (Settings, and System info) might give a clue on what happened.

I don’t know the code that well, but I believe the check is at open time. The server DB opens right away to even show you your backup jobs screen. Some specific backup DB is probably not opened until required. There is a backup taken (as mentioned earlier), and one change in recent versions is now the backup file name includes the name of the file it came from (as opposed to just a date, leaving one a difficult guess).

Differens in database files, what are they?
Include original database filename when saving database backup. #3792

So while I’m not making a firm promise, Duplicati doesn’t AFAIK handle multiple DB versions at once. Its solution is to upgrade (as required) on first use (unless it does everything in sight – look at your results).

So I seriously doubt a late DB version complaint is possible.

What’s definitely possible is that the backup of the old DB taken at version update time will grow obsolete rapidly as new backups occur. Trying to revert to it as part of a downgrade will mean losing new versions. Possibly this is what you were getting at. There’s a small time window on a given backup to go back, and possibly testing a less critical backup first when you do a major version jump would be the safer method.

Good.

I didn’t get update, I got notification about the new beta, and even though I have already dismissed the notification, I got it again, possibly after rebooting. I already updated my release channel setting, but that didn’t help with the notification, I still got it. This insistence may trick some users, I warn you.

The “on first use” part here is what worries me, so let me rephrase my question: does the “first use” of each db happen before doing anything else or can it happen when Duplicati has already done something for the job run at hands?

If I cannot trust that all db version checks precede all other steps, then I will have to try the code excavation route, but I don’t particularly like being pushed into it.

It certainly can’t begin anything complex such as a backup job run without a database.
An operation that uses the database upgrades the database before use (as was said).

Possibly it could do an export because that info is in server database, not the backup.
There are probably other things one could do with a job (maybe delete it?) with no DB.
Chasing down all things that don’t need the DB is pointless because it doesn’t matter.

This being a user forum, there are probably many questions that nobody can answer.
Developers don’t know all things either, and there’s no lack of work they’re needed for.

If the previous request for information from “Settings” and “System info” still fits, please supply that, e.g.

* ServerVersion : 2.0.4.22
* ServerVersionName : - 2.0.4.22_canary_2019-06-30
* ServerVersionType : Canary
* StartedBy : Tray icon
* BaseVersionName : 2.0.4.22_canary_2019-06-30
* DefaultUpdateChannel : Canary

(mine isn’t very interesting at the moment, but the BaseVersion and the current ServerVersion may differ)

Ideally, reproduce the issue. In any event, please note if you have ever used Beta or Beta channel recently prior to the unwanted notification. AFAIK you’re only supposed to get notifications per your channel setting.

Detailing your exact history may also help. Notifications appear to be persisted in the server database, so possibly a stale one was shown again when it should have been purged, e.g. at time of channel changing.

There does not appear to be any widespread issue, because there are no other reports of this happening.

I don’t know if it “fits”, but my system info is exactly the same as yours, on both of my machines (Windows and Ubuntu). I have not updated Duplicati via the web UI, I have always downloaded and run the installation package.
Unfotunately, I cannot remember if I have received the notification on both of my machines. Possibly not, as their histories are different. Originally, I installed a beta on both. Then a new beta came, and its installation failed on Windows. After that, I had beta on Ubuntu and canary on Windows. I never updated the canary until I bought a new Windows machine and scrapped the old. First I installed the same canary as before on the new machine, so it does not have a beta in its history. After that, I upgraded both machines to the latest canary to have the same version on both. And then I got notification of the latest beta and tried it, failed, and reverted back to the latest canary on both machines.
So probably I keep getting the beta notification on my Ubuntu machine only. Somehow, it is difficult to remember that kind of detail, and although I meticulously keep notes of the installations I make, receiving a notification is not something I record in my notes.

EDIT: I just realized that there is a mismatch between Duplicati system info and settings. On both machines, I have selected Experimental as the default channel (but haven’t downgraded from canary to experimental yet). Duplicati System Settings does not reflect this; it tells I have Canary as default setting on both machines.