Efficiently Split a Large Backup

How-To
1

I have one backup in my “stable” that is quite large. It consists of two main sources on a Windows 7 machine: a \users\ folder, and a 4TB USB drive that has about 1.1TB in use currently.

Together the sources measure 1.45 TB. They currently occupy about 960GB on the destination, probably mostly thanks to deduplication between the \users\ and the USB. Which is what I’d expect. The system is used by a photographer and all camera “rolls” are stored on the USB, while a subset, actively being worked on, will be in the \users\ folders.

Considering the OneDrive deal for 5TB in 1TB slices, I was thinking about splitting this backup so that I could synch each individual partition to a TB slice of a OneDrive Home account. I’m afraid of not having room for growth if I do it all together; starting off the use of a new storage service at 95% capacity strikes me as asking for trouble.

I have more than enough room on the local destination to make a full copy of the 960GB backup. Even two copies, if that would help.

The thing I most want to minimize with this effort is the amount of work I have to make the source machine do to pull it off.

What would be the group’s suggestions for how to proceed?

2

Since it sounds like you have the space, I’d try this:

  1. copy your destination file into a 2nd directory
  2. export your first backup job and import it into a 2nd job but point it to the new destination (so now you have two of everything)
  3. edit one job to be one set of data and the other to be the remainder of the data

This will let you keep your history while splitting your jobs. Each job will consider the contents of the other job as having been deleted locally and will clean them out of the backup history as your “keep” rules come into play (unless they’re “forever”).

Note that when the cleanup happens there will likely end up being lots of “small” archive files so expect a lot of downloads / uploads as the archives are shrunk and consolidated. But in the end, you should have two separated jobs with separated histories.

Please note that I don’t know of any way to JOIN two jobs while keeping histories on both of them. (While it’s probably technically possible, it would require quite a bit fancy DB work.)

1 Like
3

Thanks. Starting to try that now. It will take a couple days.

4

Quesion: if I remove the DBPath element from the .json file, will that stimulate the creation of a new database? Is that what I want? I am pretty sure I don’t want to reuse the same database…

5

When you import the exported version of your existing copy you SHOULD get a “Use existing database?” message like this one:

image
image717×257 11.3 KB

Clicking the “No” button should give you a new database name - but no actual database file yet.

There are two ways to handle this:

  1. Go into “Database → Repair” to have to the local database be created (from the remote index files, can take a while)
  2. Copy the existing job database file into the name of the new database file (this is my choice, as it avoids the recreate step)
6

Yes - all running smoothly now!

Looks like it will take a while, though, to realize the space savings. For the time being, each copy is the same size as the original. But I know that the older versions just have to age out.

1 Like
7

If you don’t want to wait, you could try the purge command…

1 Like