Release: 2.0.4.23 (beta) 2019-07-14

2.0.4.23-2.0.4.23_beta_2019-07-14

This update only contains warnings to inform users on Amazon Cloud Drive that the service is discontinued.
The update will warn users running or configuring a backup stored on Amazon Cloud Drive.

This does NOT affect other Amazon services such as S3.
If you do not use Amazon Cloud Drive, you can skip this update.

For more information on the changes, please see:

Is this release only to add the ACD warnings, or are other Canary/Experimental features rolled up into this as well?

This only contains the Amazon Cloud warnings.

The version 2.0.4.23 beta is faulty, and I think you should pull it back.

Yesterday, I was making backups with the previous version, 2.0.4.22 canary, and had no problem with it. Then in the evening I noticed a new beta had arrived, and installed it on both of my machines. This morning, I found Duplicati backups having succeeded fine on my Ubuntu machine, but on Windows, it was screaming red telling me for each job that

“The database has version 9 but the largest supported version is 8. This is likely caused by upgrading to a newer version and then downgrading. If this is the case, there is likely a backup file of the previous database version in the folder C:\Users\haltia\AppData\Local\Duplicati.”

A brief look at the location %LOCALAPPDATA%\Duplicati\ revealed that the job-specific sqlite files had not been touched since yesterday’s succesfull runs, only the file Duplicati-server.sqlite had (on both machines, I run Duplicati locally, using a local disk as target).

My knee-jerk reaction was to look up a command in the manual to rebuild the file somehow, but then I took a step back and started reasoning. I have never downgraded Duplicati on my Windows machine. On Ubuntu I have, and it didn’t cause any problem. Anyway, it occurred to me that sine I had only yesterday the exact previous version working right, so maybe the quick fix would be to uninstall the latest and reinstall version 2.0.4.22 back. And lo and behold, it works fine with the database!

I cannot understand why the problem occurs only one of the machines, why not the both. So, I would appreciate it if you could clarify the error message to state whether “previous database version” refers to SQLite version or your own schema version. I do assume you mean schema version, but adding the word “schema” to the error message would go a long way to clarify it, even if you consider it over-clarification.

Anyway, I think a problem of this kind could confuse too many of you devoted users, and pulling back the faulty version would be the best first action.

I have to agree.
I upgraded from 2.0.4.22_canary to 2.0.4.23_beta on Windows 10 and I have the same database problem. I reinstalled back 2.0.4.22_canary and everything works.

The latest beta is nothing more or less than the previous beta (2.0.4.5) with an added warning message for anyone using the Amazon Cloud Drive. This service will be discontinued next month, so users (especially in the beta channel) should be informed soon about this shutdown.

After the latest beta, a lot of new features and improvements have been added. These changes require modifications to the database structure, but have not been tested enough to be merged to the beta version.

This is why the database of the latest beta have a lower version number than the latest experimental and canary versions. “Updating” from a recent canary or experimental version will effectively be a downgrade to respectively version 2.0.4.4 and 2.0.4.2.

As stated by @kenkendk there’s no need to update to this version, unless you’re in the beta channel and want to be informed by the upcoming closure of the ACD service:

Oh, I didn’t think before installation :slight_smile:

Your answer reveals a problem much bigger than the glitch I reported. You are saying that users should know not to expect the latest beta version contains the features of the latest interim release. I see. That has not been adequately emphasized, especially to us whose native language is not English. But that is not the major problem. The BIG problem is that the database version has NEVER been indicated on the download page. Despite my native language, if I had seen the database (schema?) version mentioned each time a new version was released, I would have been weary about installing the new beta. This is one of the most fundamental pieces of information that usually is always given to users. Your download page doesn’t even contain a link to full list of changes per release.

You have to start giving the most basic information to your users.

Oh, and the other question then: how come I don’t have the same problem on my Ubuntu machine? I just ran two of the backups just to check, and I don’t get the error. I re-checked and yes, I have the latest beta (2.0.4.23) on the Ubuntu machine. How is it possible that the problem does not appear?

Your Ubuntu machine was either running the previous beta/experimental, or just a canary that was old enough to be on the same database version as the “new” beta.

If your Windows machine and Ubuntu machine had both had the same Canary version both would behave the same.

I am not sure how we should communicate this, but the idea is that each channel is independent. That means, if you are on beta, just stay there and you will get the most stabe/tested updates. Likewise goes for experimental, if you stay on it, all works.

Generally, the canary will have the latest fixes and updates, but occasionally issues. Normally, you would be able to change from a canary to a beta, but for this release we had a short time window to warn Amazon Cloud Drive users. A problem was detected with the FTP provider in the canary, so we decided to delay all other changes than the warning, so even tough the version numbers increment, they do not follow each other.

This is a design decision, and we need to be able to run different channels so we do not delay or prevent updates to the canary while releasing a beta.

We could be more clear about the version numbers, such that they convey “newer than”. My idea with the number was to simply increment from 2.0.4.5, but 2.0.4.6 was taken, so I just went to the to the next number.

I could have used 2.0.4.6, but that would also be confusing as there would be two different builds with the same number but one with a canary tag and one with a beta tag.

You are the first person to request this information :slight_smile:

But I think you have a point. We should perhaps force increment the third value (i.e. we should be on 2.0.5.x for the canary and 2.0.4.x for the beta) to indicate database schema changes.

It has only been slightly problematic when people want to downgrade a canary, but here the problem occurs because you changed the update channel (or manually installed the beta).

I have not seen you even attempting to communicate this. It is so contrary to the practices in other software projects that you really need to make it clear.

How about the first official release, can one upgrade to it from any branch or just the beta branch?

It is very much a normal practice to inform users about database upgrades in every release they occur - so normal that I expect it to be the default.

Close, but not quite. I had for a long time had the latest beta, 2.0.4.5, on the Ubuntu machine (on Windows, that was the plan as well, but that version just could not be installed, so I chose the latest Canary instead, and never had regrets). However, I had installed the latest canary on the both machines recently, because I was trouble-shooting a problem on the Ubuntu machine. I even did a downgrade during the trouble-shooting, but ended up having the latest canary and once I had figured out what the problem was (my own mistake), I managed to run the problematic backup a few times. And then I installed this new beta. And then backups succeeded fine, no problem at all.

The only explanation I can come up with is that after installing the beta, I may have forgotten that the Ubuntu machine was still running the previous version. The Windows installer forces me to shut down the currently running Duplicati processes, but on Ubuntu that does not happen, at least not when I make the installation from a downloaded deb package.

Should I create a bug report or feature request for shutting down the processes also on Linux?

I’m not sure if that difference is a feature or a bug. Linux usually allows updates to software without interference with running software. Meanwhile windows often require restarts to apply.

Forcing it on both would probably be more user friendly but I could see it annoying some experienced Linux users.

Forcing is not necessary. Showing the processes to user to remind that they are running the old version would be enough.

Could be a fair compromise

When using Firefox Nigthly, a switch to the stable release is a downgrade as well. We do not have nightly builds (my fault) but the same logic applies to canary builds.

Can you give an example of where such information should be placed? Maybe a specific wording that would have made it clear to you?

The experiemental and beta branches for sure. The canary branch may have drifted (it usually does) in the time we wait for last-minute feedback on the releases.

Can you give an example of other software that does this? I am unsure how to present the information, so I would like to see.

My experience is from published releases, which are in production use. Most cases I can remember are software I have needed at work. Thinking of OpenSource software that I use at home, I can mention GnuCash, which is mature enough to not have database upgrades very often. When it does, users are informed loudly about it in release announcements, release notes, and … I was trying to check the download page, but the site seems unreachable right now. The release annnouncements and releases notes are prominent enough, though. I always check them for incompatibility information.

I can understand that while Duplicati is still in development phase, my nagging may seem nit-picking. Now that I think of it, it is possible that when I first installed Duplicati2, I did read and understandd that the channels are separate. But that was a few years back, and after that, nothing has reminded me about it. Originally, I had planned to use beta releases only, but then 2.0.4.5 failed to install on Windows, and somehow I got derailed into choosing canary instead of experimental.

The best way to to remind the user is the same place where you describe the changes in each release, briefly or not. In your case, that seems to be the download page. The minimum I would expect to see there is the database version. That alone would have alerted me to check again the difference of the channels.

I think this is where Duplicati wants to go, and I’ve heard a wish for more frequent betas than current plan.

Perhaps it’s time to think about how to juggle competing objectives:

  • More frequent betas, also avoiding pushes for “one more fix”
    Is a sense of upcoming beta good, or a contribution-blocker?
    I’m not bold enough to suggest a specific calendar schedule

  • Ability to hot-fix betas for things that can’t wait until next beta
    Outside disruptions happen, and “test escapes” can as well

  • Use usage reporter data to track canary/experimental pickup
    A faster beta cycle will mean less time to hope for field tests

  • Encourage and develop better-trained volunteers to help test
    Ideally they also know how to collect data, and maybe debug
    Rare problems seen only in the field are extremely hard to fix

  • Simple enough version methods to not confuse contributors
    Duplicati can’t train everybody the way a big company might

  • A few contributors who know Git well enough to do backports
    Probably don’t want to get excessive about parallel branches
    Latest beta gets supported, but that might include a backport

  • Release numbers that don’t contain unexpected surprises…
    I wish 2.0.4.23 came out 2.0.4.5.1, but we’re out of numbers
    There was also the idea of third field going up on DB change
    I’m wondering when second field will ever go up. Wasted 0?
    Leave gaps in number sequence for future use, e.g. 2.0.4.6?

  • Document release warnings and oddities as discussed here
    DB version info is good, but also say what it means to users

At the moment, it looks like the way three channels emerge from one master is to keep canary open, then lead developer does an experimental as a beta pre-release, then a beta happens. Experimental isn’t open to all changes, as seen here. It’s not clear how open canary is. Huge changes and DB changes might risk preventing an experimental-beta restart-from-canary if that becomes needed. I don’t recall if it’s happened.

This is all part of growing pains. Duplicati is (slowly) becoming higher quality, and to maintain its level while moving forward (can’t afford paralysis from fear of breakages) is the challenge. Is this worth a discussion? Technically it belongs in Developer category but I’ll start here because there’s an experienced VCS person.

Personally, I’ve found that branches and version control can be a headache. Maybe smaller steps are best for a project this size. While there’s a core of regular contributors, there are even more less frequent ones.