Backup failed with many "expected missings"

1

One of my backups has started failing. repair did nothing. Here is an extract of the log with the beginning and end. I omited the many in between on almost all the versions .

2025-03-16 10:48:40 -04 - [Error-Duplicati.Library.Main.Operation.BackupHandler-FatalError]: Fatal error
UserInformationException: Unexpected difference in fileset version 292: 6/6/2024 3:30:00 AM (database id: 834), found 314513 entries, but expected 314966

Unexpected difference in fileset version 290: 6/8/2024 3:30:00 AM (database id: 836), found 319920 entries, but expected 320373

Unexpected difference in fileset version 285: 6/13/2024 3:36:08 AM (database id: 841), found 338948 entries, but expected 339399

Unexpected difference in fileset version 284: 6/14/2024 3:30:00 AM (database id: 842), found 339466 entries, but expected 339917
‘’‘’‘’‘’‘’ skipping to end
Unexpected difference in fileset version 11: 3/5/2025 2:30:00 AM (database id: 1115), found 727192 entries, but expected 732798

Unexpected difference in fileset version 9: 3/7/2025 2:30:00 AM (database id: 1117), found 728776 entries, but expected 734384

Unexpected difference in fileset version 8: 3/8/2025 2:30:00 AM (database id: 1118), found 730931 entries, but expected 736539

Unexpected difference in fileset version 7: 3/9/2025 3:30:00 AM (database id: 1119), found 731900 entries, but expected 737508

Unexpected difference in fileset version 6: 3/10/2025 3:30:00 AM (database id: 1120), found 732731 entries, but expected 738339

Unexpected difference in fileset version 5: 3/11/2025 3:30:00 AM (database id: 1121), found 733877 entries, but expected 739485

Unexpected difference in fileset version 4: 3/12/2025 3:30:00 AM (database id: 1122), found 735298 entries, but expected 740906

Unexpected difference in fileset version 3: 3/13/2025 3:30:00 AM (database id: 1123), found 723532 entries, but expected 729140

Unexpected difference in fileset version 2: 3/14/2025 3:30:00 AM (database id: 1124), found 724437 entries, but expected 730045

Unexpected difference in fileset version 1: 3/15/2025 3:30:00 AM (database id: 1125), found 724883 entries, but expected 730491

2

Do you have any idea of what caused this? Like what happened before?

Could you create a bugreport database and share it? Then I can maybe get closer to the reason for this.

Also, what operating system is this on and what Duplicati version?

3

I opened an issue (#6051)

The Enviornment is

Duplicati - 2.0.9.101_canary_2024-06-27
Ubuntu 24.04
Microsoft OneDrive v2

This does 4 dailys a day that have worked for 9 months, and now one of the 4 has started to fail.

4

Arh, well in that case, I will write the same information here:
Can you create a bugreport database and share it?

You can PM me on the forum if you prefer not sharing the link publicly.

5

I was unclear on what create a bugreport database meant. I thought that meant open an issue.

How do I create a bugreport database.

6

Sorry, I was not being clear on that.
To make a bugreport database via the UI, expand the backup configuration and click “Create bugreport…”:

Screenshot 2025-03-19 at 16.16.03
Screenshot 2025-03-19 at 16.16.031102×746 50.7 KB

This will create a copy of the local database and scrub it of personal information (paths etc).
After it completes, you get a zip file that you can check to see if it contains any sensitive information.
Once you see that it is correctly scrubbed, you can upload it somewhere and share it with me via a PM.

7

Not so easy. The bugreport creation fails with “Attempted to write a stream that is larger than 4GiB without setting the zip64 option”. The presumably partial zip file left in /tmp was at 887MB when it failed. I don’t have any control of setting any option to zip since duplicati is invoking it.

8

I think this is the Advanced option that the error message suggests:

  --zip-compression-zip64 (Boolean): Toggle ZIP64 support
    The ZIP64 format is required for files larger than 4GiB. Use this option to toggle it.
    * default value: False
9

I’m just picking “Create bug report” from the gui under Reporting: . It doesn’t give any way to give a zip option.

10

Assuming you have a job, put it on the job options. That applies to many operations, not just backup runs. For example, that’s where you can put Restore options as well.

Job → Edit → Options → Advanced options

image
image659×279 21.2 KB

11

With the zip64 option enabled, the create bugreport ends with no error
message. Now I’m not clear where the report is or what to look for. I
see one zip file in /tmp. Is this the file I should share?

Archive: /tmp/dup-69b3972e-fce2-4171-81c4-81391f490dbe.zip
Length Method Size Cmpr Date Time CRC-32 Name

9518194688 Defl:N 2918670237 69% 2025-03-20 10:53 5ba0da6c
log-database.sqlite
902 Defl:N 491 46% 2025-03-20 11:44 7fa28585 system-info.txt

9518195590 2918670728 69% 2 files
Archive: /tmp/dup-69b3972e-fce2-4171-81c4-81391f490dbe.zip
Length Method Size Cmpr Date Time CRC-32 Name

9518194688 Defl:N 2918670237 69% 2025-03-20 10:53 5ba0da6c
log-database.sqlite
902 Defl:N 491 46% 2025-03-20 11:44 7fa28585 system-info.txt

9518195590 2918670728 69% 2 files

12

It’s supposed to say (and does for me)

image

When you click Download, your browser does (however it does them).

is what it should make. There should be a small system-info.txt and a (very big for this case) log-database.sqlite. Your output is kind of run together, but it looks likely.

13

I have a PR for fixing bugreports of files larger than 4 GiB.

Great call! Yes, the options are applied to the operation as well.

Not sure about the format, but the filenames log-database.sqlite and system-info.txt are the two files that are inside the zip file. If you can share this somehow, I would love to dig into it.

You can PM me with a link if you have some place where you can store it.