How do I purge an incomplete backup?

I have a job that ran 3 times without issue to (so 3 restorable versions)

After deciding it was working as expected, I increased what was included as part of the Source but at some point the job crashed with a “The channel is retired” error.

After reviewing why it might have crashed (I’m still not sure - maybe a bandwidth limit?) I realized I had included way too much in my backup to fit in the destination.

So I have pared down my source so that it will fit in the destination, but I still a lot of space (like 30G) taken up by the failed run.

I’ve pared down my Source again so it should fit in the destination, but I’m not sure how to purge the contents of the failed run so it’s not taking up useless space.

I tried running list-broken-files but it doesn’t show anything as being broken.

Hmm, compacting should take care of those 30G. Any archives uploaded on a failed run would in essence be 100% unused blocks similarly to when you delete a restore point and it’s associated files are cleaned up.

Good thought - thanks!

So I’ve got 48.7G in there right now and am kicking off a Compact now run via the job menu. :crossed_fingers:

Well, at least it didn’t take long to tell me it wouldn’t do what I want. :frowning:

Jul 23, 2018 2:28 PM: Compacting not required
Jul 23, 2018 2:28 PM: Found 0 volume(s) with a total of 0.00% wasted space (0 bytes of 55.91 GB)
Jul 23, 2018 2:28 PM: Found 3 small volumes(s) with a total size of 8.13 MB
Jul 23, 2018 2:28 PM: Found 0 fully deletable volume(s)

I double checked directly on and confirmed that about 1,880 of the files 2,020 showed up with the last run, so at least I’m not craze in thinking there’s a bunch of wasted space now.

As a last resort I assume I can just delete those files which should let me use purge-broken-files but I’m hoping there’s a “cleaner” way to handle it.

Hmm, odd that compacting doesn’t help…

Broken files should only be files that are in the dlist but that are missing part of the blocks required to restore the file. So in my understanding it shouldn’t delete anything.

Is compacting and the “deleting unwanted files” step of a backup not the same? Because if they’re different then you might be in a scenario where you have to unselect files until you can fit a new backup within your quota and get to the clean up step at the end, which is not ideal.

If I delete everything (dlist, dined, dblock) upload with the last (failed) run then when I try to run again it will complain about missing files and tell me to do a Repair. The repair should make everything right for me - but it’s a messy way to do it and prone to error (if I delete too much then I break a previous backup).

My guess is the compact isn’t working because files are actually there as they should be.

The problem as I see it is that because the job didn’t finish there’s no version associated with those files for me to point to when I tell the job to delete or purge. It’s possible that if I complete another job the “orphaned” files will be brought over to the new version and THEN I could do the cleanup, but I would also be deleting potentially GOOD backups of the new run.

So to be safe I’d have to run a backup, then right away do the purge, then right away to another backup. Even better for me would be to run the faster delete instead of purge and let the normal compact handle things later - but that only works because I have enough free space to do a few more of the smaller backups. If I had actually run out of disk space at the destination I’m not sure that option would be available.

Of course my backup isn’t critical, so I may try the “do another run and see if it includes the previous run’s orphan files” just out of curiosity…

1 Like

After reducing what was getting backed up I ran another back which finished successfully and it appears I’ve magically gone from 48.7G of used spaced on the destination down to 2.63G.

So the something in the normal backup process seems to have found the “non-versioned” destination files and cleaned them up automatically.

It looks like it took about 2 hours to do all the deletes. :slight_smile:

1 Like

Hmm, okay. That does seem to indicate that compacting does not delete unwanted “unlisted” files the same way the “deleting unwanted files” step does. I’d argue it should or that this cleanup step should be possible to trigger from the UI next to compacting.

That makes sense to me - syntactically I’d think of an “unwanted” file as something I know I don’t want (as in actively flagged as such) while an “unlisted” file is something I’m not really sure about.

I’m going to flag my last post as the “solution” but maybe @kenkendk will have some time to pop in and give his thoughts on this since it could still be a failure point if I had actually run out of destination storage space. :wink: