Removing a file(s) or folder(s) from all backups

Newbie question. New to Duplicati and new to commandline interfaces.

How do you remove a file(s) or folder(s) (including all contents and sub-folders) from all backups?

It looks like this is possible using the ‘purge’ command, but I am very confused how to use it.

I’m not sure how to run command line for Duplicati.

The user manual doesn’t explain how to do command line for a newbie.

This YouTube video is even more confusing and the only example is running a backup.

On the web UI, I see the backup job, under ‘Advanced’, has a link ‘Commandline…’ but the next page is quite overwhelming for a newbie.

In Finder, I see a program called ‘duplicati-cli’, but it seems to not do anything.

Sidenote, Time Machine makes removing a file from all backups pretty simple in a visual way. I’d like to try using Duplicati for backups though and this would be helpful.

Thanks for any help.

Yep, it is possible with the PURGE command.

I recommend using the web UI if that’s where you defined your backup jobs. You got the right idea by clicking the blue “command line…” link. On that next page, pick “purge” from the dropdown menu. Replace all the contents of the “Commandline arguments” text box with something like this:

--include=/path/to/purge/*

Then scroll to the bottom and click the “run purge command now” button.

Thank you.

For the “/path/to/purge/*”

Is the path what it would look like on the source or the destination?

What would the first level on the far left of this path be? The drive letter? ftp? Something else?

Can an entire folder be purged?

In the example you linked to, can you clarify the difference between “–include=/path/to/purge/*” and “< storage-URL > < filenames >” ?

It’s the source side.

Whatever the path looks like from Duplicati’s perspective. I got the idea you were using a Mac since you mentioned Time Machine. So no drive letter or anything.

On Windows you would use a drive letter and backslashes:

--include=D:\Path\To\Purge\*

Yep, when you use an asterisk like the above examples, all contents of those folders are purged from the backup data.

storage-URL is the backup destination. It should be populated automatically by the web UI. It’s what appears in the “target URL” box:

1

The “filenames” parameter can be used if you want to just purge a single file. If you want to purge an entire folder, you need to use the --include= option with a wildcard like I showed above. But in either case they would go in the “command line arguments” box in the above screen shot.

1 Like

This is helping, thank you.

Some follow up questions.

For the Mac’s source path, if the folder in question was a folder in a user’s Documents folders, would it look something like:

–include=/Users/macusernamehere/Documents/foldernamehere/*

If so, does that delete the contents of foldernamehere or does it also delete the folder itself from Documents?

Based on your reply, it sounds like the Target URL field shouldn’t be modified in this situation, correct?

Is there a way to test that this will work or did work? Duplicati doesn’t seem to have a way to show deleted files from backups in the same way Time Machine does. I see there is a “–dry-run” command, but I’m not sure exactly what that is going to do based on the one sentence in the user manual.

I’m not a Mac person but from a quick Google search it looks like that would be correct.

From my experience the entire “foldernamehere” folder is removed from backups and is no longer displayed at all in the folder tree when you attempt a restore.

Correct.

Yes, as you noted you can use --dry-run parameter. You’ll be able to see the job output to confirm there are no errors. Nothing actually changes so the “purged” folder will still appear in the restore folder tree of course.

You might consider creating a new backup job just to test out the purge operation so you get more familiar/comfortable with it. Make a new job, select a small number of folders, do at least one backup, then try the purge command.

This process seems to have only removed the folder from the most recent backup.

Is there something that may have caused this? How do I remove the folder from all backups?

Did you use the --version option? If you do, the purge will only affect the specified version(s). If you leave it off it should do all backup versions.

Still testing…

No, I have not used the version option.

I think one issue may be that the 2 short dashes were changed to 1 long dash below:

Question: For backups of Mac folders with a space in the folder name, how should the command be handled? A few possible options are:

/Users/macusernamehere/Documents/folder name here/*
/Users/macusernamehere/Documents/folder\ name\ here/*
/Users/macusernamehere/Documents/folder_name_here/*

Another challenge is that it takes a very long time (several minutes for each level of the backup’s folder hierarchy, so a deeply nested folder in multiple backups takes hours altogether!) to see if running a purge command has worked or not. This separate, but related, challenge seems to be under discussion already. Also, the past few attempts to open each restore version page for the backup produces this pop-up:

Error
Missing XSRF Token. Please reload the page

Pressing OK on this pop-up seems to make it go away without any further obvious issue at this point.

Another challenge is that I often run into “database is locked” errors, even though nothing is happening with the database. Usually, running a “verify” first, then running a “commandline…” seems to address this issue when it occurs.

Oh yeah, that would definitely be a problem. Try again with two short dashes.

I can’t test this myself as I don’t have a Mac, but I think escaping the spaces with a backslash should work. Enclosing the path in double quotes would probably work, too.

What version of Duplicati are you using? What browser?

This usually means you’re trying to do two things at once. The normal Duplicati scheduler only does one thiong at a time, but if it’s busy doing a backup or something and you try to do a restore or manually run some other command, you can get this message.

Looks like a simple space worked! Thanks for your help!

What is quite strange is that the date of the backup version that contained the folder I wanted purged was an earlier date than the date of the file that the purge command appeared to delete.

2.0.6.3
Firefox

Only doing one Duplicati thing at a time.