Can one 'manually' delete a backup set? SOLVED

Hi there,

I’m just wondering if it’s possible to delete a backup set from within the web interface (or elsewhere). As an example, i have three backup sets and the first is some 800mb while the other 2 are only a few mb. The reason is i changed the source location to be backed up. Can i delete the first (800mb) backup set and keep the remaining two? A third backup set would be created (if necessary) on the next run.

Cheers.

When you say “backup set” do you mean a specific dated backup like you’d see listed in the “Restore Files” interface or do you mean an entire backup job like you’d create when you click “Add backup”?

G’day JonMikeIV - i was referring to the specific dated backup like you’d see listed in the “Restore Files” interface :slight_smile:

For that I think you want the Delete command with the --version-#parameter.

Running "C:\Program Files\Duplicati 2\Duplicati.CommandLine.exe" help delete returns:

Usage: delete <storage-URL> [<options>]

  Marks old data deleted and removes outdated dlist files. A backup is deleted
  when it is older than <keep-time> or when there are more newer versions
  than <keep-versions>. Data is considered old, when it is not required from
  any existing backup anymore.

  --keep-time=<time>
    Marks data outdated that is older than <time>.
  --keep-versions=<int>
    Marks data outdated that is older than <int> versions.
  --version=<int>
    Deletes all files that belong to the specified version(s).
  --allow-full-removal
    Disables the protection against removing the final fileset

So if you wanted to delete all the “file versions” in backup version #3 you could run something like:

"C:\Program Files\Duplicati 2\Duplicati.CommandLine.exe" delete "ssh://mySSH:12345/mnt/testBackup?auth-username=me&auth-password=myPass&ssh-fingerprint=ssh-rsa%20myFinger" --version=3

Or you should be able to use the GUI “Commandline…” option something like this:
image

Edit: Not matter what method you choose to use, I’d suggest either trying it on a test backup and/or using the --dry-run parameter to make sure you’ve got all the parameters set right.

2 Likes

Thanks, i haven’t been able to get it to work. Mine is running on a test VM, with my test backups stored on the desktop. I suspect a noob pebcak error.

Here’s what i’ve attempted via the CommandLine GUI:

Backup
Target URL: /home/kubuntu/Desktop/Backups?auth-username=kubuntu&auth-password=kubuntu
Arguments: --version=3

It seems to run forever, with “Running commandline entry” text, and i eventually abort (there’s no activity suggesting files are being deleted - the backup folder is not getting smaller).

When i abort, i get this:

System.Threading.ThreadAbortException
  at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String[]& paths, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x0011a] in <118ad25945a24a3991f7b65e7a45ea1e>:0 
  at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.Action`1[T] method) [0x00009] in <118ad25945a24a3991f7b65e7a45ea1e>:0 
  at Duplicati.Library.Main.Controller.Delete () [0x00006] in <118ad25945a24a3991f7b65e7a45ea1e>:0 
  at Duplicati.CommandLine.Commands.Delete (System.IO.TextWriter outwriter, System.Action`1[T] setup, System.Collections.Generic.List`1[T] args, System.Collections.Generic.Dictionary`2[TKey,TValue] options, Duplicati.Library.Utility.IFilter filter) [0x00087] in <08be045d41054657bb406455b4cf1a83>:0 
  at (wrapper delegate-invoke) System.Func`6[System.IO.TextWriter,System.Action`1[Duplicati.Library.Main.Controller],System.Collections.Generic.List`1[System.String],System.Collections.Generic.Dictionary`2[System.String,System.String],Duplicati.Library.Utility.IFilter,System.Int32]:invoke_TResult_T1_T2_T3_T4_T5 (System.IO.TextWriter,System.Action`1<Duplicati.Library.Main.Controller>,System.Collections.Generic.List`1<string>,System.Collections.Generic.Dictionary`2<string, string>,Duplicati.Library.Utility.IFilter)
  at Duplicati.CommandLine.Program.RunCommandLine (System.IO.TextWriter outwriter, System.IO.TextWriter errwriter, System.Action`1[T] setup, System.String[] args) [0x003fb] in <08be045d41054657bb406455b4cf1a83>:0 
System.Threading.ThreadAbortException
  at Duplicati.CommandLine.Program.RunCommandLine (System.IO.TextWriter outwriter, System.IO.TextWriter errwriter, System.Action`1[T] setup, System.String[] args) [0x00586] in <08be045d41054657bb406455b4cf1a83>:0 
  at Duplicati.Server.WebServer.RESTMethods.CommandLine+<>c__DisplayClass4_0.<POST>b__0 (Duplicati.Library.Main.IMessageSink sink) [0x00030] in <a6c0c2089b9a44ec9be5057a44f12116>:0 

The error message you included was due to aborting the process, so it doesn’t provide any insight into what might (or might not) have been going on.


Suggestions

Firstly, I’d suggest trying the delete again but including the --no-auto-compact=true parameter. This turns off a bunch of behind the scenes cleanup steps that you can read more about below (if you care).

Secondly, I’d suggest that you try running the delete again and then opening a 2nd browser window to the Duplicati GUI (likely something like http://localhost:8200/ngax/index.html#/log) so you can check the “Live” log in “Information” or “Profiling” mode. This should let you know about each specific step Duplicati is taking, even if it doesn’t result in a GUI progress update.

Read on if you care about the guts going on behind the scenes that might actually be making the delete take so long.


What goes on behind the scenes

Keep in mind that Duplicati doesn’t store raw files that can just be deleted - it chops files up into smaller blocks (of size --blocksize, default 100K) and stores the blocks (if not already in a backup) in an archive file (of size --dblock-size, default 50MB) OR stores a reference to an existing same-content block (if found in an existing backup).

All these block references are stored in the local .sqlite database file, so at it’s basic level a delete is really just going through the database, finding all the block references for a particular backup version (in this case #3) and flagging them as “deleted”.

What happens next varies depending on other setting in your config, but typically if a dblock (archive) file is found to have at least X% (`–threshold, default 25%) of the blocks stored in it flagged as deleted then the archive file is downloaded, uncompressed, “deleted” flagged blocks deleted, then the remaining content re-compressed, and the resulting file uploaded.

After that, if dblock (archive) files are found to be smaller that size X (--small-file-size, default 20% of dblock-size) they are downloaded, uncompressed, merged, re-compressed into fewer ‘more fully utilized’ files, and then uploaded.

So there are a number of areas that could have caused apparent stalling in the code:

  1. if you have a LOT of records in the local sqlite file it can take a while to run all the commands identifying deleteable blacks
  2. if the flagging of blocks as deleted causes a lot of dblocks (archives) to become sparsely populated it could take a while to download, re-compress, and re-upload them
  3. if the re-compression of dblocks (archives) causes multiple dblocks to become “too small” it could take a while to download, merge, re-compress, and re-upload them
1 Like

Maybe there is a glitch that activates the password module? Try adding --passphrase=<passphrase> on a line by itself. You also need to supply the --dbpath parameter.

PEBCAK.

In previous attempts, i believe i deleted all advanced options - which include things like the passphrase, database location and whatnot.

This time i ran it keeping all those in place and left the original target URL (i also tried appending the ?username/password which didn’t help - same output):
Target URL: file:///home/kubuntu/Desktop/Backups
Arguments: --version=3

Plus all the pre-existing advanced settings which include the database name etc.

This quickly finished with:

  Listing remote folder ...
No remote filesets were deleted
No filesets matched the criteria
Return code: 0

Is it expecting a remote path and is doing something with the URL? For testing convenience, i’m doing this all on a single-machine VM which would not be a typical use-case as obviously we wouldn’t be storing our backups on our desktop :upside_down_face:

The log output (set to Live / Information) shows:

Sep 28, 2017 8:25 AM: Email sent successfully using server: smtps://smtp.gmail.com:465
Sep 28, 2017 8:25 AM: No remote filesets were deleted
Sep 28, 2017 8:25 AM: Backend event: List - Completed: (44 bytes)
Sep 28, 2017 8:25 AM: Backend event: List - Started: ()

This indicates that there is no “version 3” of the data?

What if you run the `LIST`` command? It should show you the versions.

1 Like

OMG.

So there are 3 datasets… but they are numbered 0, 1, 2.

How embarrassment! :sweat_smile:

It worked after i used --version=2.

Sorry guys, and thank you :sunny:

There’s multiple posts that solved the ‘problem’ so not sure which i should say did so, so i’ll update this one with the parameters i used and say this one.

To delete my third backup set (the big one), which is actually “2” (sets are 0, 1, 2 in a three backup set) here’s what i did:

  • Open the GUI.
  • Select the backup set.
  • Go to the Commandline section.
  • Change “command” to ‘Delete’
  • Target URL: file:///home/kubuntu/Desktop/Backups
  • Arguments: --version=2
  • Leave all the advanced options alone.
  • Click ‘run delete command now’ (bottom of screen)
1 Like