[Solved] After importing configuration and running task is it ok to "Rapair"?

I Imported configuration. Locally I have all the data that is in the cloud and also after the last backup, I added some additional data locally. When I ran imported task I got the message:
Error while running My Archive

Found 1055 remote files that are not recorded in local storage, please run repair

Screenshot: https://i.imgur.com/wBjEXnH.jpg

  1. What that exactly means? What exactly is missing in local storage?
  2. What is the best course of action? Repair?

Please hold off on a repair until we hear some more information on what you’re doing.
Did you ever do anything like copy old local database (.sqlite file) into the new config?
If not, there is a risk of repair deleting destination data to match its empty database…

What are you trying to do when doing import?

1 Like

Everything below I did on one laptop. I used Backblaze B2 as a cloud.

What I did:

  1. I set up back up job. I write down passphrase.
  2. Exported job config.
  3. I ran back-up job for several times.
  4. Updated duplicati.
  5. I ran back-up job for several times.
  6. I deleted the existing back-up job but I do not remember I checked also to delete database or not. I did not delete local files and have not renamed the main folder (so main folder is in place with same name).
  7. I deleted several files and added several files from/to my local folder.
  8. I imported backup job configuration. Added passphrase. Also checked “Import metadata” and “Save immediately” checkboxes.
  9. I ran the backup job and received:

Error while running My Archive

Found 1055 remote files that are not recorded in local storage, please run repair

It’s not clear why in step 6 the job was deleted, then brought back in step 8 from the step 2 export, however the export and import only handle the configuration (and some metadata). The local database matching the destination files needs to be brought along with those files. A new job creates a new database, but you can put the old one into the new job like this, if you still have it. I think there’s a bug that prevents deletion when the job is deleted… In this case, that might help you out. I assume that old database and destination match.

Looking inside the exported config can find the old pathname in the “DBPath” line. With luck, it’s still around, and can be placed into the restored job to match the files at destination. Probably the destination was never changed due to the error on the backup, but Job --> Show log --> Remote will show you whether it changed. Seeing list is fine, put not so bad (but might be undone). Seeing a delete might mean destination is now damaged. If it’s not damaged, it’s usually possible to slowly recreate the database out of the destination files, however finding and installing a matching database from latest backup (step 5, I hope) is the preferred path.

1 Like

I could not find database, so I think it is deleted.
Here is a screenshot of the log: https://i.imgur.com/cQogPlu.jpg There is only “list”, so I assume no data loss.
I think it is better to recreate database.

  1. To recreate database I need to push “[Recreate (delete and repair)” button in Database section and wait until recreation finishes, is this correct?
  2. My backup folder is currently ~ 25 GB and has ~7000 files in it. Previously I ran backup no more than 45 times. What you think, how long database recreation will take?
  3. Is there possibility of damaging my local files while recreating database?
  1. Yes - to recreate the database, use the “Recreate (delete and repair)” button and have some patience. :slight_smile:
  2. It’s hard to estimate how long a Recreate will take. With 7k files it likely won’t take TOO long.
  3. A database recreate will download a few files from the destination to use to rebuild the local database but it won’t touch any of your source data files at all, so they should be fine.

A few notes:

  1. @ts678 is correct in that renaming the local database from it’s old name to the new one (or re-pointing the imported job to the old database name) is likely much faster & safter than recreating the database.
  2. You can paste screenshots directly into your post - for example, here 's some of my Remote log showing some GETs as well as a LIST and a DELETE:
    image

    Note that the DELETE is deleting a no-longer needed archive from the destination, it’s not deleting anything from my local machine.

Personally, I’d suggest going to the job’s Database menu and looking at the “Local database path” field. Look in the folder referenced there and see if you have 2 .sqlite files name something like DTRMXBFMSD.sqlite (ignore the duplicati-server.sqlite file).

If you have only 2 .sqlite files and only 1 (current) backup job then most likely one file will be very small and the other relatively large. Either swap the file names OR in the Duplicati GUI change the “Local database path” field to have file name of the larger .sqlite file.

This should get your original database re-attached to your imported job.

1 Like

@ts678 and @JonMikelV

I found several database files:

Previously I had also other jobs that I deleted. How to find out which database file to use and also if that database is up-to-date to actual backup?

The backup 201812xxxxxx.sqlite files are backups made prior to a Recreate (or Repair?).

Duplicati-server.sqlite is the ‘global’ database so don’t touch that.

The one that’s named with all numbers is very odd - I haven’t see anything with that naming convention before. Personally, I’d check the newest date of the files at the destination and find the nearest newer date in the .sqlite files at the local end.

1 Like

How do I know if the database I gave the job is correct one?

  1. Duplicati will give me an error if I give incorrect (for different job) or outdated database file?
  2. Giving incorrect database will not corrupt database or backup in cloud?

That’s the tricky bit - I think @Pectojin might know of a way to tease it out of the database but in general what would happen when running a backup job after attaching a different database would be one of these:

  1. it does a backup without errors (yay - you guess correctly!)

  2. it reports a FEW missing files at the destination (oops - your database is likely NEWER than your destination, how did you manage that?)

  3. it reports a FEW extra files at the destination (oops - your database is like OLDER than your destination, it’s probably a backup or left over from an exported job)

  4. it reports a LOT of missing or extra files at the destination (oops - totally wrong database, probably from a different job)

  5. Duplicati won’t start and you eventually find some “database version” errors (oops - database from a different version of Duplicati)

Other than #1 (where it worked) most of the above cases will have the Backup job aborted with an error and suggestion for a Database Repair - DON’T DO IT. As long as you don’t run a Repair (or Recreate) you won’t be screwing anything up at either end and you can try again with a different database.

1 Like

I found the database in exported job json file as @ts678 told in the first post. I just was searching in wrong place before. It worked perfectly without any error or warning.
Thank you @ts678 and @JonMikelV for help :slight_smile:

1 Like

Also, the database file that was named with only numbers was database that was created by importing backup job from json file.

Glad to hear you got it working - and thanks for sharing about the imported job database name!

1 Like