Differens in database files, what are they?

Mikki-10 · January 21, 2019, 3:29pm

Hi

What is the difference to the database files for duplicati?

DZEUKKAGTZ.sqlite (the database file used in the “live” backup?)
DZEUKKAGTZ.backup (a backup of the file or?)
DZEUKKAGTZ.backup-1 (a backup of the backup or?)
DZEUKKAGTZ.sqlite-journal (a what?)
backup 20190110120730.sqlite (one more backup of what??)

I know DZEUKKAGTZ.sqlite is used for the backup, but what about all the “.backup” files and so on, are they save to move to save some space, because I have 131GB backup files, and 65GB “live” sqlite files, so I will like to move them to a save place.

Mikki-10 · June 3, 2019, 11:41am

Any updates to this?

kenkendk · June 3, 2019, 11:51am

The file DZEUKKAGTZ.sqlite is the main database file, and used for active backups. While running, the database needs to update, and to do this in a way that is safe from crashes (i.e. the database can still be read if the machine crashes during an update) it uses a journal, in the file DZEUKKAGTZ.sqlite-journal.

I do not recall any files being made with a .backup extensions, and certainly not the .backup-1.

It should just take the database name and add a timestamp:

github.com

duplicati/duplicati/blob/master/Duplicati/Library/SQLiteHelper/DatabaseUpgrader.cs#L258


{

    // In some cases (mostly test setups)

    // the database upgrades can happen within a second of each other

    // causing the upgrades to fail. This scheme adds up to 15 seconds

    // delay, making room for multiple rapid upgrade calls

    var backupfile = string.Empty;

    for (var i = 0; i < 10; i++)

    {

        backupfile = System.IO.Path.Combine(

            System.IO.Path.GetDirectoryName(sourcefile),

            Strings.DatabaseUpgrader.BackupFilenamePrefix + " " + (DateTime.Now + TimeSpan.FromSeconds(i * 1.5)).ToString("yyyyMMddhhmmss", System.Globalization.CultureInfo.InvariantCulture) + ".sqlite");



        if (!System.IO.File.Exists(backupfile))

            break;

    }



    try

    {

        if (!System.IO.Directory.Exists(System.IO.Path.GetDirectoryName(backupfile)))

            System.IO.Directory.CreateDirectory(System.IO.Path.GetDirectoryName(backupfile));

Which should give something like DZEUKKAGTZ 20190110120730.sqlite as the file name.

The other names sound more like something Finder or Explorer would make.

Mikki-10 · June 3, 2019, 1:27pm

Its running as a docker from linuxserver/duplicati on my Unraid, the docker is based on Ubuntu 16.04 LTS, so it might be the docker that is doing fun things?

So i can asume that the files .backup and .backup-1 are safe to move… maybe…

Thanks for the update

kenkendk · June 3, 2019, 7:54pm

Unless you specifically request that Duplicati uses those database file (i.e. setting --dbpath=...) it will not look for them. So from a Duplicati perspective they are safe to remove.

drwtsn32 · June 3, 2019, 8:32pm

on my systems running 2.0.4.5, the automatic backups are called “backup timestamp.sqlite” - the original filename is nowhere in the backup filename. Was that fixed after 2.0.4.5 to work as you describe?

ts678 · June 3, 2019, 8:51pm

The posted source link is in master, so I think it still drops the original name (which is maybe not good). BackupFilenamePrefix in the code quote seems to be “backup”, resulting in the names you’re seeing.

drwtsn32 · June 3, 2019, 9:31pm

Thanks for digging into it a bit. I agree, I think it’d be good to include the original filename when construction the backup.

kenkendk · June 4, 2019, 6:59am

Yes I forgot that part. The idea was at some point that you could name the backup with --backup-name. The UI should set this name.

I agree we should change it to use the original filename, as that is more telling than the current “backup”.

drwtsn32 · June 5, 2019, 4:42am

Correct me if I’m wrong, but I’m thinking the whole “try for 10 times at 1.5 second increments” is because the original part of the database filename is NOT included in the backup. So when you first install a newer version, there are at least 2 databases (main and one backup set) that need to be backed up.

So if we fix the backup filename so that it includes the original database filename, we won’t need that kludge.

What do you think?

drwtsn32 · June 6, 2019, 4:01am

Submitted a pull request…