"Deleting Unwanted Files" takes hours

Hello,

1st, thanks for this great program ! :slight_smile:

I have an issue with it though : “Deleting Unwanted Files” takes 2 or 3 hours to complete, about 90% of the time, if not more.

I’m using OneDrive as backup storage. I did a database repair which didn’t change anything.
I’m running 2.0.1.61_experimental_2017-05-30 , I had the same issue with the previous version (I forgot which one exactly).

I just checked the live log, here the last lines of a mix of information & profiling :

    Aug 13, 2017 8:27 AM: Backend event: Put - Started: duplicati-ba5ea5ec9fd6148e8a01f56ba2034e1d6.dblock.7z.aes (9.13 MB)
    Aug 13, 2017 8:27 AM: Backend event: Get - Completed: duplicati-b3a4e0bd490d94d618ca914e7733831c0.dblock.7z.aes (7.17 MB)
    Aug 13, 2017 8:27 AM: Backend event: Get - Started: duplicati-b3a4e0bd490d94d618ca914e7733831c0.dblock.7z.aes (7.17 MB)
    Aug 13, 2017 8:27 AM: Backend event: Delete - Completed: duplicati-i73ff57f6cc534141a6486e61761478c8.dindex.7z.aes (21.01 KB)
    Aug 13, 2017 8:27 AM: Backend event: Delete - Started: duplicati-i73ff57f6cc534141a6486e61761478c8.dindex.7z.aes (21.01 KB)
    Aug 13, 2017 8:27 AM: Backend event: Delete - Completed: duplicati-b386c41a1ec144199997ef01fc6c5ddc3.dblock.7z.aes (8.36 MB)
    Aug 13, 2017 8:27 AM: Backend event: Delete - Started: duplicati-b386c41a1ec144199997ef01fc6c5ddc3.dblock.7z.aes (8.36 MB)
    Aug 13, 2017 8:27 AM: Starting - RemoteOperationDelete
    Aug 13, 2017 8:27 AM: ExecuteReader: SELECT "C"."Name", "B"."Name", "B"."Hash", "B"."Size" FROM "IndexBlockLink" A, "RemoteVolume" B, "RemoteVolume" C WHERE "A"."IndexVolumeID" = "B"."ID" AND "A"."BlockVolumeID" = "C"."ID" AND "B"."Hash" IS NOT NULL AND "B"."Size" IS NOT NULL took 00:00:00.000
    Aug 13, 2017 8:27 AM: Starting - ExecuteReader: SELECT "C"."Name", "B"."Name", "B"."Hash", "B"."Size" FROM "IndexBlockLink" A, "RemoteVolume" B, "RemoteVolume" C WHERE "A"."IndexVolumeID" = "B"."ID" AND "A"."BlockVolumeID" = "C"."ID" AND "B"."Hash" IS NOT NULL AND "B"."Size" IS NOT NULL
    Aug 13, 2017 8:27 AM: RemoteOperationNothing took 00:00:00.000
    Aug 13, 2017 8:27 AM: Starting - RemoteOperationNothing
    Aug 13, 2017 8:27 AM: RemoteOperationPut took 00:00:02.643
    Aug 13, 2017 8:27 AM: Backend event: Put - Completed: duplicati-i5025d950b5e548d495a760a86c132170.dindex.7z.aes (21.65 KB)
    Aug 13, 2017 8:27 AM: Uploaded 21.65 KB in 00:00:02.6286627, 8.24 KB/s
    Aug 13, 2017 8:27 AM: Backend event: Put - Started: duplicati-i5025d950b5e548d495a760a86c132170.dindex.7z.aes (21.65 KB)
    Aug 13, 2017 8:27 AM: Starting - RemoteOperationPut
    Aug 13, 2017 8:27 AM: RemoteOperationPut took 00:01:35.547
    Aug 13, 2017 8:27 AM: Backend event: Put - Completed: duplicati-b8ebeb853d8c44fc5b1daefa84f3abf95.dblock.7z.aes (7.25 MB)
    Aug 13, 2017 8:27 AM: Uploaded 7.25 MB in 00:01:35.5400534, 77.75 KB/s
    Aug 13, 2017 8:25 AM: Backend event: Put - Started: duplicati-b8ebeb853d8c44fc5b1daefa84f3abf95.dblock.7z.aes (7.25 MB)
    Aug 13, 2017 8:25 AM: ExecuteScalarInt64: INSERT INTO "Remotevolume" ("OperationID", "Name", "Type", "State", "Size", "VerificationCount", "DeleteGraceTime") VALUES (?, ?, ?, ?, ?, ?, ?); SELECT last_insert_rowid(); took 00:00:00.000
    Aug 13, 2017 8:25 AM: Starting - ExecuteScalarInt64: INSERT INTO "Remotevolume" ("OperationID", "Name", "Type", "State", "Size", "VerificationCount", "DeleteGraceTime") VALUES (?, ?, ?, ?, ?, ?, ?); SELECT last_insert_rowid();
    Aug 13, 2017 8:25 AM: ExecuteScalarInt64: INSERT INTO "Remotevolume" ("OperationID", "Name", "Type", "State", "Size", "VerificationCount", "DeleteGraceTime") VALUES (?, ?, ?, ?, ?, ?, ?); SELECT last_insert_rowid(); took 00:00:00.000
    Aug 13, 2017 8:25 AM: Starting - ExecuteScalarInt64: INSERT INTO "Remotevolume" ("OperationID", "Name", "Type", "State", "Size", "VerificationCount", "DeleteGraceTime") VALUES (?, ?, ?, ?, ?, ?, ?); SELECT last_insert_rowid();
    Aug 13, 2017 8:25 AM: Starting - RemoteOperationPut
    Aug 13, 2017 8:25 AM: ExecuteNonQuery: DROP TABLE IF EXISTS "DelVolSetIds-63250F3F760E24409CE876B1E4621316" took 00:00:00.000
    Aug 13, 2017 8:25 AM: Starting - ExecuteNonQuery: DROP TABLE IF EXISTS "DelVolSetIds-63250F3F760E24409CE876B1E4621316"
    Aug 13, 2017 8:25 AM: ExecuteNonQuery: DROP TABLE IF EXISTS "DelBlockSetIds-63250F3F760E24409CE876B1E4621316" took 00:00:00.000
    Aug 13, 2017 8:25 AM: Starting - ExecuteNonQuery: DROP TABLE IF EXISTS "DelBlockSetIds-63250F3F760E24409CE876B1E4621316"
    Aug 13, 2017 8:25 AM: ExecuteNonQuery: DELETE FROM "DelVolSetIds-63250F3F760E24409CE876B1E4621316" took 00:00:00.000
    Aug 13, 2017 8:25 AM: Starting - ExecuteNonQuery: DELETE FROM "DelVolSetIds-63250F3F760E24409CE876B1E4621316"

Thanks in advance for any help ! :slight_smile:

I think this is caused by using 7z files:

Try setting --no-auto-compact, or switch to zip.

We are considering labelling 7z as “unstable” as we have not yet had the issues fixed.

1 Like

Thanks ! I added the “–no-auto-compact” option in the project settings and for 3 days in a row it ran very fast again !