UsageReport SQL running even if disabled?

Thanks to @Pectojin’s keen eye at Slow backup -especially deleting unwanted files and creating of bug report it appears some UsageReport SQL is being executed even if it is set to “None / disabled”.

Changing the “Usage statistics” setting doesn’t seem to affect the 6 UsageReport SQL related log entries below.

Is there a reason this is being run even though usage reporting is disabled?

2018-03-09 18:09:47Z - Profiling: Starting - ExecuteNonQuery: CREATE TEMPORARY TABLE "UsageReport-8142E816E75DE04C820340C49CCE443D" AS SELECT "VolumeID" AS "VolumeID", SUM("ActiveSize") AS "ActiveSize", SUM("InactiveSize") AS "InactiveSize", MAX("Sortime") AS "Sorttime" FROM (SELECT "A"."ActiveSize" AS "ActiveSize",  0 AS "InactiveSize", "A"."VolumeID" AS "VolumeID", CASE WHEN "B"."Sorttime" IS NULL THEN 0 ELSE "B"."Sorttime" END AS "Sorttime" FROM (SELECT SUM("Block"."Size") AS "ActiveSize", "Block"."VolumeID" AS "VolumeID" FROM "Block", "Remotevolume" WHERE "Block"."VolumeID" = "Remotevolume"."ID" AND "Block"."ID" NOT IN (SELECT "Block"."ID" FROM "Block","DeletedBlock" WHERE "Block"."Hash" = "DeletedBlock"."Hash" AND "Block"."Size" = "DeletedBlock"."Size") GROUP BY "Block"."VolumeID" ) A LEFT OUTER JOIN (SELECT "VolumeID" AS "VolumeID", MIN("Sorttime") AS "Sorttime" FROM (SELECT "Block"."VolumeID" AS "VolumeID", "Fileset"."Timestamp" AS "Sorttime" FROM "Fileset", "FilesetEntry", "File", "BlocksetEntry", "Block" WHERE "FilesetEntry"."FileID" = "File"."ID" AND "File"."BlocksetID" = "BlocksetEntry"."BlocksetID" AND "BlocksetEntry"."BlockID" = "Block"."ID" AND "Fileset"."ID" = "FilesetEntry"."FilesetID"  UNION SELECT "Block"."VolumeID" AS "VolumeID", "Fileset"."Timestamp" AS "Sorttime" FROM "Fileset", "FilesetEntry", "File", "BlocksetEntry", "Block", "Metadataset" WHERE "FilesetEntry"."FileID" = "File"."ID" AND "File"."MetadataID" = "Metadataset"."ID" AND "Metadataset"."BlocksetID" = "BlocksetEntry"."BlocksetID" AND "BlocksetEntry"."BlockID" = "Block"."ID" AND "Fileset"."ID" = "FilesetEntry"."FilesetID" ) GROUP BY "VolumeID" ) B ON "B"."VolumeID" = "A"."VolumeID"  UNION SELECT 0 AS "ActiveSize", SUM("Size") AS "InactiveSize", "VolumeID" AS "VolumeID", 0 AS "SortScantime" FROM "DeletedBlock" GROUP BY "VolumeID"  UNION SELECT 0 AS "ActiveSize", 0 AS "InactiveSize", "Remotevolume"."ID" AS "VolumeID", 0 AS "SortScantime" FROM "Remotevolume" WHERE "Remotevolume"."Type" = ? AND "Remotevolume"."State" IN (?, ?) AND "Remotevolume"."ID" NOT IN (SELECT "VolumeID" FROM "Block") ) GROUP BY "VolumeID" 
2018-03-09 18:09:48Z - Profiling: ExecuteNonQuery: CREATE TEMPORARY TABLE "UsageReport-8142E816E75DE04C820340C49CCE443D" AS SELECT "VolumeID" AS "VolumeID", SUM("ActiveSize") AS "ActiveSize", SUM("InactiveSize") AS "InactiveSize", MAX("Sortime") AS "Sorttime" FROM (SELECT "A"."ActiveSize" AS "ActiveSize",  0 AS "InactiveSize", "A"."VolumeID" AS "VolumeID", CASE WHEN "B"."Sorttime" IS NULL THEN 0 ELSE "B"."Sorttime" END AS "Sorttime" FROM (SELECT SUM("Block"."Size") AS "ActiveSize", "Block"."VolumeID" AS "VolumeID" FROM "Block", "Remotevolume" WHERE "Block"."VolumeID" = "Remotevolume"."ID" AND "Block"."ID" NOT IN (SELECT "Block"."ID" FROM "Block","DeletedBlock" WHERE "Block"."Hash" = "DeletedBlock"."Hash" AND "Block"."Size" = "DeletedBlock"."Size") GROUP BY "Block"."VolumeID" ) A LEFT OUTER JOIN (SELECT "VolumeID" AS "VolumeID", MIN("Sorttime") AS "Sorttime" FROM (SELECT "Block"."VolumeID" AS "VolumeID", "Fileset"."Timestamp" AS "Sorttime" FROM "Fileset", "FilesetEntry", "File", "BlocksetEntry", "Block" WHERE "FilesetEntry"."FileID" = "File"."ID" AND "File"."BlocksetID" = "BlocksetEntry"."BlocksetID" AND "BlocksetEntry"."BlockID" = "Block"."ID" AND "Fileset"."ID" = "FilesetEntry"."FilesetID"  UNION SELECT "Block"."VolumeID" AS "VolumeID", "Fileset"."Timestamp" AS "Sorttime" FROM "Fileset", "FilesetEntry", "File", "BlocksetEntry", "Block", "Metadataset" WHERE "FilesetEntry"."FileID" = "File"."ID" AND "File"."MetadataID" = "Metadataset"."ID" AND "Metadataset"."BlocksetID" = "BlocksetEntry"."BlocksetID" AND "BlocksetEntry"."BlockID" = "Block"."ID" AND "Fileset"."ID" = "FilesetEntry"."FilesetID" ) GROUP BY "VolumeID" ) B ON "B"."VolumeID" = "A"."VolumeID"  UNION SELECT 0 AS "ActiveSize", SUM("Size") AS "InactiveSize", "VolumeID" AS "VolumeID", 0 AS "SortScantime" FROM "DeletedBlock" GROUP BY "VolumeID"  UNION SELECT 0 AS "ActiveSize", 0 AS "InactiveSize", "Remotevolume"."ID" AS "VolumeID", 0 AS "SortScantime" FROM "Remotevolume" WHERE "Remotevolume"."Type" = ? AND "Remotevolume"."State" IN (?, ?) AND "Remotevolume"."ID" NOT IN (SELECT "VolumeID" FROM "Block") ) GROUP BY "VolumeID"  took 00:00:00.194
2018-03-09 18:09:48Z - Profiling: Starting - ExecuteReader: SELECT "A"."Name", "B"."ActiveSize", "B"."InactiveSize", "A"."Size" FROM "Remotevolume" A, "UsageReport-8142E816E75DE04C820340C49CCE443D" B WHERE "A"."ID" = "B"."VolumeID" ORDER BY "B"."Sorttime" ASC 
2018-03-09 18:09:48Z - Profiling: ExecuteReader: SELECT "A"."Name", "B"."ActiveSize", "B"."InactiveSize", "A"."Size" FROM "Remotevolume" A, "UsageReport-8142E816E75DE04C820340C49CCE443D" B WHERE "A"."ID" = "B"."VolumeID" ORDER BY "B"."Sorttime" ASC  took 00:00:00.000
2018-03-09 18:09:48Z - Profiling: Starting - ExecuteNonQuery: DROP TABLE IF EXISTS "UsageReport-8142E816E75DE04C820340C49CCE443D" 
2018-03-09 18:09:48Z - Profiling: ExecuteNonQuery: DROP TABLE IF EXISTS "UsageReport-8142E816E75DE04C820340C49CCE443D"  took 00:00:00.000
1 Like

Troubleshooting this a bit more, I think I’m seeing that the usage report SQL queries do not run if I restart the Duplicati server after disabling the setting.

In that case it may just be an issue with the global setting only being applied on startup.

Odd - I’m looking a the code and it looks like the SQL is part of GetWastedSpaceReport in Duplicati\Library\Main\Database\LocalDeleteDatabase.cs.

Assuming I have that right then I think it’s just an unfortunate naming overlap between:

  • UsageReporting (as in logging usage)
  • UsageReport (as in wasted space usage in a dblock)

In that case what I’m seeing may just be that Duplicati doesn’t feel like checking if it should compact right now.

It started creating a UsageReport temporary table again now (after it actually deleted files from the backend).

So I’m almost ready to buy the UsageReporting vs UsageReport explanation :wink: