Downgrading / reverting to a lower version

Nuts - you’re right, it’s unclear based on name alone whether the backup is for Duplicati-server.sqlite or [job].sqlite.

Could you please update this with the latest database versions?

@AimoE, thanks for pointing out that database b versions have changed. I haven’t had time to keep up with the latest versions but I’ll try to take a look in a few days.

Note that the first post should be a wiki so if you already know the release and database numbers you can update it yourself by using the Edit button at the bottom right of the post. :slight_smile:

But I don’t know the versions, how could I? That is exactly why I hope you would update the info.

Before you publish the first official version of Duplicati, you need to learn to provide users the basic compatibility information, such as db version changes, with the releases (plus link to full list of changes).

Sorry, I thought you had requested an updated list because you knew a database version had changed.

I figured you had learned it in one of the same way I do:

  • review pull requests
  • see it mentioned in a release announcement
  • experience or hear of a database specific issue with a downgrade.

If you find something about Duplicati lacking please feel free to join in and help us improve it! If it’s not happening now it’s likely because we either don’t know how or don’t have the time to do it. :slight_smile:

You’ll find the context for my request here: Release: 2.0.4.23 (beta) 2019-07-14 - #6 by kees-z

I used the information from that post to update the wiki. Feel free to change it if you think it unclear or incorrect.

But you didn’t add the information I was asking for. The error message I got from the latest beta mentioned database versions 8 and 9. Your posting stops at version 7. Which Duplicati version came with db version 8? Which came with 9?

My guess (on a thread you were on) is below. Would you consider donating some hours to explore this? Basically, it’s a whole lot of clicking plus record-keeping. Retroactive testing is probably too much to ask. Previous note here also points out that there is no downgrade test suite, so precision data loses value…

Error accessing database version 9 plus a note that 2.0.4.5 does 8, and I’m not sure I want to go further, however I suspect the table at top could use confirmation, and there are also two different DBs involved.

I’m thinking v2.0.4.13-2.0.4.13_canary_2019-01-29 through (so far) v2.0.4.22-2.0.4.22_canary_2019-06-30 however if you’re not testing bleeding-edge canary releases, v2.0.4.21-2.0.4.21_experimental_2019-06-28 might be what you had.

Basically, to be the most helpful, one would say where a DB got added on each of the release channels. Databases for the server DB and the backup DB go independently, but either one can hurt a downgrade.

https://github.com/duplicati/duplicati/tree/master/Duplicati/Server/Database

https://github.com/duplicati/duplicati/tree/master/Duplicati/Library/Main/Database

That did occur to me too at some point. Unfortunately, despite its lengthy wording, the error message I received does not specify which db is at question. Maybe I should place a bug report on that in your long queue of bugs.

So the only way to find out what I need to know is try downgrading and see if it succeeds. I am principally interested to know if I have a chance to downgrade from latest canary to latest experimental and then maybe some day get it upgraded to beta.

Can you promise that if any one of the two dbs isn’t compatible after a downgrade, I will get an error message at first attempt to run a job, not down the line later?

Not exactly. This whole topic is sort of in violation of the it-hasn’t really-been-tested caveat, but at least the click-source-tree plan (don’t even need to sign into anything) can tell you when DB version WILL complain. As a side note, I can’t recall any downgrades that didn’t work due to something other than the DB versions.

Based on what I wrote (assuming I wrote/re-read it right), latest experimental and latest canary should play fine on DB version. Presumably move forward to next beta would also work. Downgrade to 2.0.4.5 will not.

At the moment, latest
Canary is v2.0.4.22-2.0.4.22_canary_2019-06-30
Experimental v2.0.4.21-2.0.4.21_experimental_2019-06-28, and IIRC the actual code is basically rebuild of v2.0.4.20-2.0.4.20_canary_2019-06-25 (sometimes an Experimental is custom-blended, but this was not).

Because downgrade from latest canary to latest experimental is only a small step, possibly your reason is getting off the Canary upgrade treadmill (which by design is bleeding-edge). If so, you can instead change channel to Experimental in Settings, overriding the default of Canary channel you have if you install Canary.

Getting information about your setup (specifically System info tab) might help before you change things, especially since you gave a thought in some topic that you got an update in violation of the channel plan. Info from these two screens I mention (Settings, and System info) might give a clue on what happened.

I don’t know the code that well, but I believe the check is at open time. The server DB opens right away to even show you your backup jobs screen. Some specific backup DB is probably not opened until required. There is a backup taken (as mentioned earlier), and one change in recent versions is now the backup file name includes the name of the file it came from (as opposed to just a date, leaving one a difficult guess).

Differens in database files, what are they?
Include original database filename when saving database backup. #3792

So while I’m not making a firm promise, Duplicati doesn’t AFAIK handle multiple DB versions at once. Its solution is to upgrade (as required) on first use (unless it does everything in sight – look at your results).

So I seriously doubt a late DB version complaint is possible.

What’s definitely possible is that the backup of the old DB taken at version update time will grow obsolete rapidly as new backups occur. Trying to revert to it as part of a downgrade will mean losing new versions. Possibly this is what you were getting at. There’s a small time window on a given backup to go back, and possibly testing a less critical backup first when you do a major version jump would be the safer method.

Good.

I didn’t get update, I got notification about the new beta, and even though I have already dismissed the notification, I got it again, possibly after rebooting. I already updated my release channel setting, but that didn’t help with the notification, I still got it. This insistence may trick some users, I warn you.

The “on first use” part here is what worries me, so let me rephrase my question: does the “first use” of each db happen before doing anything else or can it happen when Duplicati has already done something for the job run at hands?

If I cannot trust that all db version checks precede all other steps, then I will have to try the code excavation route, but I don’t particularly like being pushed into it.

It certainly can’t begin anything complex such as a backup job run without a database.
An operation that uses the database upgrades the database before use (as was said).

Possibly it could do an export because that info is in server database, not the backup.
There are probably other things one could do with a job (maybe delete it?) with no DB.
Chasing down all things that don’t need the DB is pointless because it doesn’t matter.

This being a user forum, there are probably many questions that nobody can answer.
Developers don’t know all things either, and there’s no lack of work they’re needed for.

If the previous request for information from “Settings” and “System info” still fits, please supply that, e.g.

* ServerVersion : 2.0.4.22
* ServerVersionName : - 2.0.4.22_canary_2019-06-30
* ServerVersionType : Canary
* StartedBy : Tray icon
* BaseVersionName : 2.0.4.22_canary_2019-06-30
* DefaultUpdateChannel : Canary

(mine isn’t very interesting at the moment, but the BaseVersion and the current ServerVersion may differ)

Ideally, reproduce the issue. In any event, please note if you have ever used Beta or Beta channel recently prior to the unwanted notification. AFAIK you’re only supposed to get notifications per your channel setting.

Detailing your exact history may also help. Notifications appear to be persisted in the server database, so possibly a stale one was shown again when it should have been purged, e.g. at time of channel changing.

There does not appear to be any widespread issue, because there are no other reports of this happening.

I don’t know if it “fits”, but my system info is exactly the same as yours, on both of my machines (Windows and Ubuntu). I have not updated Duplicati via the web UI, I have always downloaded and run the installation package.
Unfotunately, I cannot remember if I have received the notification on both of my machines. Possibly not, as their histories are different. Originally, I installed a beta on both. Then a new beta came, and its installation failed on Windows. After that, I had beta on Ubuntu and canary on Windows. I never updated the canary until I bought a new Windows machine and scrapped the old. First I installed the same canary as before on the new machine, so it does not have a beta in its history. After that, I upgraded both machines to the latest canary to have the same version on both. And then I got notification of the latest beta and tried it, failed, and reverted back to the latest canary on both machines.
So probably I keep getting the beta notification on my Ubuntu machine only. Somehow, it is difficult to remember that kind of detail, and although I meticulously keep notes of the installations I make, receiving a notification is not something I record in my notes.

EDIT: I just realized that there is a mismatch between Duplicati system info and settings. On both machines, I have selected Experimental as the default channel (but haven’t downgraded from canary to experimental yet). Duplicati System Settings does not reflect this; it tells I have Canary as default setting on both machines.

You seem to know more about the current database versions than I do at the moment. This is exactly the scenario I had in mind when making the post a wiki so anybody could update it - so that as my time available to give to Duplicati varies (currently very low) others could add their knowledge.

All that being said, @ts678 is right - pretty much every downgrade issue I recall had been related to DB versions, and right now there’s no way easy for a general user to know what version(s) are in use for their installation.

Ideally, the updater & installer should check database version and alert the user that continuing to install / update to the incoming version maybe result in “instability”.

However, that’s likely a fair bit of effort for a “power user” edge case. Perhaps a good start would be to include database version on the system info page (it’s not there now, right?).

Then the wiki could be updated to use that and we avoid this issue of versions bring out of sync across upgrade streams.

Thank you. This so far the most reasonable response. Would you like me to add feature request for it or do you want to do it yourself? How about modification to the error message that only says “database” instead of “server database” or “client database” or some such?

EDIT: It might actually be a good idea to have a separate support tool that checks the db versions (and maybe some other things as need arises). Eventually, having the feature in Duplicati itself is the target, but it takes time until it gets into an experimental version, and beta even later. Besides, with a separate tool, one would be able to check backed-up db versions before choosing which Dulicati to install.

“System info” is later than ideal (but nothing so far is ideal, including work of manually keeping info here).

My main “System info” concern is that by the time you can get there, your database has been upgraded. Home screen needs to open that database to see the jobs, and at opening it, it’s upgraded if it’s needed. Certainly one who kept notes on what they had before can notice to downgrade fast, but few may notice.

Additionally it’s not any help for downgrade safety forecasting. Are you sure you won’t click a few hours?

https://github.com/duplicati/duplicati/tree/master/Duplicati/Server/Database/Database%20schema
https://github.com/duplicati/duplicati/tree/master/Duplicati/Library/Main/Database/Database%20schema

Use the Branch selector to go through Tags (Duplicati release names), and note results for each channel. Finish by updating the post at top. Optionally stop by every now and then if you notice it gets out of date…

We should also try to release note such change, but you’re not talking to anybody who writes those notes.

Unless the separate support tool is just dropped into GitHub to download, it follows Duplicate release plan. Also, tool would need to either manually be told all the info (obtained manually) or maybe build could find it. Someone who knows git better than I do might be able to comment on ways to maintain a historical trail…

Due to the huge backlog, your best short-term bet is to click, look, keep records, and post the result here.

EDIT: So below is what backup (not server) DB version 9 looks like. One can guess at change points by more clicking on buttons like “History” or “Blame”, but the safest approach is to just look at every version.

This bounces me back to my question how can I avoid producing a corrupt result after downgrade.

You say that just opening the web UI upgrades “my” database. That must be the server database? And a client database is upgraded when a backup job starts up, before anything else happens in that run?

In that case, reversing back to the downgrade case, the server db version is checked when I open the web UI? And client database version is checked at beginning of a job I run? This would mean that I have a safe procedure to check the situation after a downgrade. First I open the web UI, and if it doesn’t give me an error, the server database version is OK. I then run a backup job (any one of them) and if that job doesn’t fail with db version check, then the downgrade is succesful?

Correct.

For backup, yes as said above including source links if you wish to confirm, or to check other operations. Restore (as expected) looks very similar. It needs the database, opens it early, and, upgrades as needed. Every operation must be researched individually. Some need the database but some do not. It takes time.

Language is loose again. Some jobs don’t even use the database. Backup and restore open, check, use.

I believe your safety test is roughly right, but I haven’t tested. One loophole may be that because upgrade seemingly is done as needed, if you safety-checked a backup that had never run on the new release, you might find no issue with it just because it had never been upgraded. So trying a truly used backup is good. Minimal check might dip a toe into a Restore. If you can see versions and files to pick, backup DB is open.

My language was loose also on what safe means. I would regard the procedure safe if it satisfies two conditions: For one, if no errors come up, I can then safely use the downgraded version. For second, if an error does come up, I can upgrade back to the version I started with, and continue using it, assuming that the failed downgrade has not corrupted my databases or the backup files in the target filesystem.
I can minimize damage by creating a test job before downgrade. This doesn’t help with the server database, but covers the other half.

Since I run all my backup jobs daily, I can rely on all databases having the same version.