Unable to recreate database, fails with "SQLite error cannot rollback - no transaction is active"

I’ve been trying to rebuild the database of my Fedora Server backup, but it keeps failing with “SQLite error cannot rollback - no transaction is active”.

I’ve tried “repair”, “recreate” and also “delete” then “repair” making sure the database file really deleted, and still the same. I’ve check disk space including increasing the size of /tmp and there is plenty. It always reaches the final fileset when I watch the verbose server logging. Any ideas?

Here’s the final error log:

11 Feb 2020 10:17: The operation Repair has failed with error: SQLite error cannot rollback - no transaction is active
{"ClassName":"Mono.Data.Sqlite.SqliteException","Message":"SQLite error cannot rollback - no transaction is active","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":"  at Mono.Data.Sqlite.SQLite3.Reset (Mono.Data.Sqlite.SqliteStatement stmt) [0x00096] in <5b5fb53ffa6a4c24837c4c8b9fbd2a0f>:0 
  at Mono.Data.Sqlite.SQLite3.Step (Mono.Data.Sqlite.SqliteStatement stmt) [0x00046] in <5b5fb53ffa6a4c24837c4c8b9fbd2a0f>:0 
  at Mono.Data.Sqlite.SqliteDataReader.NextResult () [0x00129] in <5b5fb53ffa6a4c24837c4c8b9fbd2a0f>:0 
  at Mono.Data.Sqlite.SqliteDataReader..ctor (Mono.Data.Sqlite.SqliteCommand cmd, System.Data.CommandBehavior behave) [0x00051] in <5b5fb53ffa6a4c24837c4c8b9fbd2a0f>:0 
  at (wrapper remoting-invoke-with-check) Mono.Data.Sqlite.SqliteDataReader..ctor(Mono.Data.Sqlite.SqliteCommand,System.Data.CommandBehavior)
  at Mono.Data.Sqlite.SqliteCommand.ExecuteReader (System.Data.CommandBehavior behavior) [0x00006] in <5b5fb53ffa6a4c24837c4c8b9fbd2a0f>:0 
  at Mono.Data.Sqlite.SqliteCommand.ExecuteNonQuery () [0x00000] in <5b5fb53ffa6a4c24837c4c8b9fbd2a0f>:0 
  at Mono.Data.Sqlite.SqliteTransaction.IssueRollback (Mono.Data.Sqlite.SqliteConnection cnn) [0x00012] in <5b5fb53ffa6a4c24837c4c8b9fbd2a0f>:0 
  at Mono.Data.Sqlite.SqliteTransaction.Rollback () [0x00008] in <5b5fb53ffa6a4c24837c4c8b9fbd2a0f>:0 
  at Mono.Data.Sqlite.SqliteTransaction.Dispose (System.Boolean disposing) [0x0001e] in <5b5fb53ffa6a4c24837c4c8b9fbd2a0f>:0 
  at System.Data.Common.DbTransaction.Dispose () [0x00000] in <6fbeaa5ac81e46008e9fcd2981e30d56>:0 
  at Duplicati.Library.Main.Operation.RecreateDatabaseHandler.DoRun (Duplicati.Library.Main.Database.LocalDatabase dbparent, System.Boolean updating, Duplicati.Library.Utility.IFilter filter, Duplicati.Library.Main.Operation.RecreateDatabaseHandler+NumberedFilterFilelistDelegate filelistfilter, Duplicati.Library.Main.Operation.RecreateDatabaseHandler+BlockVolumePostProcessor blockprocessor) [0x00f14] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Operation.RecreateDatabaseHandler.Run (System.String path, Duplicati.Library.Utility.IFilter filter, Duplicati.Library.Main.Operation.RecreateDatabaseHandler+NumberedFilterFilelistDelegate filelistfilter, Duplicati.Library.Main.Operation.RecreateDatabaseHandler+BlockVolumePostProcessor blockprocessor) [0x00037] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Operation.RepairHandler.RunRepairLocal (Duplicati.Library.Utility.IFilter filter) [0x000ba] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Operation.RepairHandler.Run (Duplicati.Library.Utility.IFilter filter) [0x00012] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Controller+<>c__DisplayClass18_0.<Repair>b__0 (Duplicati.Library.Main.RepairResults result) [0x0001c] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String[]& paths, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x0026f] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Controller.RunAction[T] (T result, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x00007] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Controller.Repair (Duplicati.Library.Utility.IFilter filter) [0x0001a] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Server.Runner.Run (Duplicati.Server.Runner+IRunnerData data, System.Boolean fromQueue) [0x003ad] in <e46664114891401e8f4656ccf8aa21a6>:0 ","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":null,"HResult":-2147467259,"Source":"Mono.Data.Sqlite"}

And these are the messages leading up to it:

11 Feb 2020 10:17: The operation Repair has failed with error: SQLite error cannot rollback - no transaction is active
11 Feb 2020 10:17: Running Repair took 0:03:28:36.377
11 Feb 2020 10:17: Recreate database for repair took 0:03:28:36.375
11 Feb 2020 10:17: ExecuteNonQuery: PRAGMA optimize took 0:00:00:00.039
11 Feb 2020 10:17: Starting - ExecuteNonQuery: PRAGMA optimize
11 Feb 2020 10:16: CommitRecreatedDb took 0:00:00:00.000
11 Feb 2020 10:16: Starting - CommitRecreatedDb

Ok trying to recreate again after a reboot, this time keeping an eye on space on all volumes and also monitoring RAM, but now it’s telling me my database is full!! What is it wanting now?

Mono.Data.Sqlite.SqliteException (0x80004005): Insertion failed because the database is full

database or disk is full
  at Mono.Data.Sqlite.SQLite3.Reset (Mono.Data.Sqlite.SqliteStatement stmt) [0x00096] in <5b5fb53ffa6a4c24837c4c8b9fbd2a0f>:0 
  at Mono.Data.Sqlite.SQLite3.Step (Mono.Data.Sqlite.SqliteStatement stmt) [0x00046] in <5b5fb53ffa6a4c24837c4c8b9fbd2a0f>:0 
  at Mono.Data.Sqlite.SqliteDataReader.NextResult () [0x00129] in <5b5fb53ffa6a4c24837c4c8b9fbd2a0f>:0 
  at Mono.Data.Sqlite.SqliteDataReader..ctor (Mono.Data.Sqlite.SqliteCommand cmd, System.Data.CommandBehavior behave) [0x00051] in <5b5fb53ffa6a4c24837c4c8b9fbd2a0f>:0 
  at (wrapper remoting-invoke-with-check) Mono.Data.Sqlite.SqliteDataReader..ctor(Mono.Data.Sqlite.SqliteCommand,System.Data.CommandBehavior)
  at Mono.Data.Sqlite.SqliteCommand.ExecuteReader (System.Data.CommandBehavior behavior) [0x00006] in <5b5fb53ffa6a4c24837c4c8b9fbd2a0f>:0 
  at Mono.Data.Sqlite.SqliteCommand.ExecuteNonQuery () [0x00000] in <5b5fb53ffa6a4c24837c4c8b9fbd2a0f>:0 
  at Duplicati.Library.Main.Database.ExtensionMethods.ExecuteNonQuery (System.Data.IDbCommand self, System.Boolean writeLog, System.String cmd, System.Object[] values) [0x0005e] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Database.ExtensionMethods.ExecuteNonQuery (System.Data.IDbCommand self, System.String cmd, System.Object[] values) [0x00000] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Database.LocalRecreateDatabase.FindMissingBlocklistHashes (System.Int64 hashsize, System.Int64 blocksize, System.Data.IDbTransaction transaction) [0x000aa] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Operation.RecreateDatabaseHandler.DoRun (Duplicati.Library.Main.Database.LocalDatabase dbparent, System.Boolean updating, Duplicati.Library.Utility.IFilter filter, Duplicati.Library.Main.Operation.RecreateDatabaseHandler+NumberedFilterFilelistDelegate filelistfilter, Duplicati.Library.Main.Operation.RecreateDatabaseHandler+BlockVolumePostProcessor blockprocessor) [0x00f3e] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Operation.RecreateDatabaseHandler.Run (System.String path, Duplicati.Library.Utility.IFilter filter, Duplicati.Library.Main.Operation.RecreateDatabaseHandler+NumberedFilterFilelistDelegate filelistfilter, Duplicati.Library.Main.Operation.RecreateDatabaseHandler+BlockVolumePostProcessor blockprocessor) [0x00037] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Operation.RepairHandler.RunRepairLocal (Duplicati.Library.Utility.IFilter filter) [0x000ba] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Operation.RepairHandler.Run (Duplicati.Library.Utility.IFilter filter) [0x00012] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Controller+<>c__DisplayClass18_0.<Repair>b__0 (Duplicati.Library.Main.RepairResults result) [0x0001c] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String[]& paths, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x0026f] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Controller.RunAction[T] (T result, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x00007] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Controller.Repair (Duplicati.Library.Utility.IFilter filter) [0x0001a] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Server.Runner.Run (Duplicati.Server.Runner+IRunnerData data, System.Boolean fromQueue) [0x003ad] in <e46664114891401e8f4656ccf8aa21a6>:0 

The disk/volumes look fine, even my increase of /tmp is surviving a reboot:

FilesystemSize  Used Avail Use% Mounted on
devtmpfs164M     0  164M   0% /dev
tmpfs180M   92K  180M   1% /dev/shm
tmpfs180M  1.2M  179M   1% /run
tmpfs180M     0  180M   0% /sys/fs/cgroup
/dev/mapper/fedora_maggie-root   20G   16G  4.2G  80% /
tmpfs500M  144K  500M   1% /tmp
/dev/sda2976M  198M  712M  22% /boot
/dev/sda1200M  7.8M  193M   4% /boot/efi
lisa.star-one.co.uk:/MAGGIE     5.5T  4.3T  1.2T  79% /srv/nfs
overlay20G   16G  4.2G  80% /var/lib/containers/storage/overlay/ed72035b2044617cbc8a82abfbd54a9c14edf811cf432cbb829be70130b
f8629/merged
overlay20G   16G  4.2G  80% /var/lib/containers/storage/overlay/c3c4d95c0edd26e86e4cda527c2cbb9edbb5097d0caf8521c76e4622911
1b904/merged
shm63M     0   63M   0% /var/lib/containers/storage/overlay-containers/6de962453043b2961ee2b16cce2966c7a1fc6f049938e814
d97f9c221fafdd7a/userdata/shm
shm63M     0   63M   0% /var/lib/containers/storage/overlay-containers/471f83cd2e4853d1b4df04449972939798030200dfac6e57
f92362145218e217/userdata/shm
overlay20G   16G  4.2G  80% /var/lib/containers/storage/overlay/0c75b01ec0277576d6c8dfab9096179d9c7ac173a272eb0412d4980782c
284c8/merged
shm63M     0   63M   0% /var/lib/containers/storage/overlay-containers/8cd3fbe87b1e9b5ceede62cb28642e361340768efb8b845c
44af7af0f8432eba/userdata/shm
overlay20G   16G  4.2G  80% /var/lib/containers/storage/overlay/b427ff590f257770dcb982ccb0dfff31bc48f865a1555ea04cc61cb4c69
e1558/merged
overlay20G   16G  4.2G  80% /var/lib/containers/storage/overlay/9a9aa037b653d6466f8eeed7b690724770d6504ec9ef54de0630f4f95c8
09326/merged
tmpfs36M     0   36M   0% /run/user/0

Ok, so maybe it was still the /tmp as increased it again to 2GB and I saw it go past 500MB briefly, will see what happens at the end of this recreate run…

So that got me further and 2G will now be my default, and it did report some errors/warnings that it logged it would fix. I reran the repair and no issues. I then ran a verify and got this:

Duplicati.Library.Interface.UserInformationException: Unexpected difference in fileset version 16: 12/04/2019 15:00:02 (database id: 5), found 44290 entries, but expected 44321
  at Duplicati.Library.Main.Database.LocalDatabase.VerifyConsistency (System.Int64 blocksize, System.Int64 hashsize, System.Boolean verifyfilelists, System.Data.IDbTransaction transaction) [0x0036e] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Operation.TestHandler.Run (System.Int64 samples) [0x0009f] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Controller+<>c__DisplayClass30_0.<Test>b__0 (Duplicati.Library.Main.TestResults result) [0x0001c] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String[]& paths, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x0026f] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.Action`1[T] method) [0x00009] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Library.Main.Controller.Test (System.Int64 samples) [0x0004b] in <45f00521e1e84c2c83156b62530c732c>:0 
  at Duplicati.Server.Runner.Run (Duplicati.Server.Runner+IRunnerData data, System.Boolean fromQueue) [0x00423] in <e46664114891401e8f4656ccf8aa21a6>:0 

I think I will just delete version 16 and be done with it.

Deleted about 10 more versions before it was finally sorted, no idea how much historically backed up files I lost - that will teach me not to set up backing up the job databases like I do on my other servers.

Which version are you running? I haven’t felt the need to do any database backups with the latest beta, 2.0.5.1. The bug that can trigger “unexpected differences” was resolved in this beta release.

2.0.5.101_canary_2020-01-23

You should not experience new instances of the “unexpected difference” issue with that version, so hopefully this problem will never return for you. If it does, please let us know.

Note that if your backups were in that state, merely upgrading to 2.0.5.x wouldn’t solve the issue. But as mentioned you shouldn’t have any new occurrences of that issue.

Some of the versions I deleted were dated after the release I have installed, a few from just a week or so ago, I’m not sure what that means against what you are stating.