Duplicati - 2.0.5.101 - Failed to connect: SQL logic error no such table: LogData

gawkla · February 17, 2020, 10:59pm

Duplicati version : 2.0.5.101
Operating system : Windows 7 64bit
Backend : SMB network drive mounted under Windows network drive

Description

Today I meet strange issue.
After upgrade to last build (Duplicati - 2.0.5.101) I can’t perform one of my backup tasks. Only one - all rest are working fine. In this one:

I can’t see logs - “Failed to connect: SQL logic error no such table: LogData”
I can’t repair, recreate database

I tried to delete local database, recreate, repair all options. Even delete backup task and import it again. Still she same wrong behaviour. Any ideas what can be wrong?

Screenshots

I have tried to recreate database, delete latest backup versions - still no progress. Issue still exists. Can anybody help me?

canove · February 17, 2020, 11:04pm

I had this erros some time ago, it was a “Run before” script on my backup set that wasn’t present on the file system anymore.

You already tried this?

gawkla · February 17, 2020, 11:11pm

yes, I have no “run before” scrip attached. This is very simple backup config: very few parameters, (no exclusion, no filters, no scripts)

ts678 · February 18, 2020, 4:31pm

Welcome to the forum @gawkla

The exact sequence is not very clear – not that unusual when testing things unless one takes notes.

When did the “Unexpected difference in fileset” happen? What might prior build have been? A Beta?
Your history of updates might be visible in Windows File Explorer, if you used the built-in autoupdate.

C:\ProgramData\Duplicati\updates
C:\Users\<Duplicati user account>\AppData\Local\Duplicati\updates

“Unexpected difference” is a consistency error, in your case in a 2019 backup that’s likely your oldest. Generally the most successful cure is to delete the named version, but sometimes the issue moves.

Without the database (did you save a copy or do you see plausible backups in the Database folder?), there are no versions to delete. Without a database at all, you get a complaint about logs because the logs were in the database that may have been deleted. Is there an empty file at the database path?

Does the backup contain old file history that is very valuable? I usually advise people not to have that. Sometimes the easiest way out is to delete the database from the web UI, delete backup files, restart.

There’s definitely more exploring that could be done, and also special tools if you want to restore now.

gawkla · February 18, 2020, 11:26pm

Hello ts678, hello Team
Update was made from v2.0.4.37 to v2.0.5.101. The most strange thing is that 7 of my 8 backup tasks works without issues. Only one is problematic. It really looks like issue is tied with last successful backup (“Unexpected difference in fileset version 1: 2020-01-27 15:39:32 (database id: 41), found 115134 entries, but expected 116808”). Since this issue happened - no further backup was made.
Now it is quite hard to give detailed description step by step.
As I wrote: I’ve tried delete/import backup task, delete last (or even 35-41 “versions”) but nothing helps.

Locally I have no database backup. But I have few copies on external USB hard-drive or even (I hope) in different duplicati backup-task. If it is necessary I can try to restore previous database version. I have tried simple “replace” database file tied with this problematic task from old backup but nothing changed.

It is not very important to save all my backuped data. I can delete everything and start from scratch but if anyone would help I can cooperate in investigation to resolve this issue and made duplicati better

ts678 · February 19, 2020, 12:46am

That’s new info. The screenshot from the original post showed issue at version 40 from 2019-10-28.
Or maybe this was a case of it moving (as it can). I’m not sure which complaint was the original one.

“Unexpected difference in fileset version 1” is a pretty common starting spot for some problems that damage the database but somehow aren’t detected on the same backup, but instead at start of next.

I’m still not quite sure where things are. If you’re deleting versions, you have a DB, so don’t have the topic title’s SQL logic error no such table: LogData I assume. Regarding old databases, they generally aren’t useful unless they’re near the problem. They are supposed to track the destination’s contents, and this change every backup, and even more if Compacting files at the backend happens.

Did an “Unexpected difference” happen on Recreate or a Backup? If on Recreate, it follows this logic:

github.com

duplicati/duplicati/blob/dbb69595947687786df15fe1e3b7e3cd109ef891/Duplicati/Library/Main/Operation/RecreateDatabaseHandler.cs#L517-L537


      
          {

              Logging.Log.WriteInformationMessage(LOGTAG, "RecreateCompletedCheckingDatabase", "Recreate completed, verifying the database consistency");

          

              //All done, we must verify that we have all blocklist fully intact

              // if this fails, the db will not be deleted, so it can be used,

              // except to continue a backup

              m_result.EndTime = DateTime.UtcNow;

          

              using (var lbfdb = new LocalListBrokenFilesDatabase(restoredb))

              {

                  var broken = lbfdb.GetBrokenFilesets(new DateTime(0), null, null).Count();

                  if (broken != 0)

                      throw new UserInformationException(string.Format("Recreated database has missing blocks and {0} broken filelists. Consider using \"{1}\" and \"{2}\" to purge broken data from the remote store and the database.", broken, "list-broken-files", "purge-broken-files"), "DatabaseIsBrokenConsiderPurge");

              }

          

              restoredb.VerifyConsistency(m_options.Blocksize, m_options.BlockhashSize, true, null);

          

              Logging.Log.WriteInformationMessage(LOGTAG, "RecreateCompleted", "Recreate completed, and consistency checks completed, marking database as complete");

          

              restoredb.RepairInProgress = false;

          }

and if on Recreate, it might also be fixable by altering the destination, especially if version is a 0 or 1.

Somewhat more normal would be if the message above about list-broken-files helps, e.g. if you do in Using the Command line tools from within the Graphical User Interface. You can change the Command dropdown and clear the Commandline arguments box, then run that and see if it has anything to say.

gawkla · February 19, 2020, 1:40pm

Hi all. maybe there is some uncleared things.
Issue described and visible on screenshoot happened after upgrade from v2.0.4.37 to v2.0.5.101

My comment “It really looks like issue is tied with last successful backup” is current status. I have checked how it looks just now and it looks like screenshot below.
This is situation after trying many repair options, delete and recreate database, delete backup job and import it again from xml, also after try of “move database”.
Now it looks like on the picture below. Database has 0 bytes on disk, There is still message “Failed to connect: SQL logic error no such table: LogData”.

The one thing what I don’t try to delete this backup job, delete all files form destination and start from scratch.
:

gawkla · February 19, 2020, 1:42pm

2020-01-27 is possible date when I made upgrade

ts678 · February 19, 2020, 2:17pm

It’s unclear which screenshot this means. Is that the one supplied with that post or some other one?

So it happened by itself before any of the many repair options tried? Was that a scheduled backup?

Error popups are not self-clearing. You use the Dismiss button if you like. Seeing the popup doesn’t reveal what you were doing when the popup arose. There were “many repair options”. Was that the result of something done (such as a LogData error is expected for empty DB), or is it original issue?

list-broken-files wasn’t tried that I hear, but it probably won’t like a completely empty database.

How about using the Dismiss button on the most recently posted red error popup now that it’s seen? These aren’t self-dismissing, and I’m still trying to get clarity on when that particular message arose.

Trying Recreate again would get a cleaner start, and find out whether Recreate said that. Also using Viewing the Duplicati Server Logs in another tab while Recreate runs might reveal some better logs. About → Show log → Live → Information might be a reasonable setting, but level can be changed.

gawkla · March 4, 2020, 10:37am

It’s unclear which screenshot this means. Is that the one supplied with that post or some other one?

First screenshot (with date 2019-10-28)

So it happened by itself before any of the many repair options tried?

yes, by itself

Was that a scheduled backup?

yes

list-broken-files wasn’t tried that I hear, but it probably won’t like a completely empty database.
was tried, result below:

Server logs show no additional info: despite “fatal error”. Probably I will delete this task and all data and will start from scratch.