Something I think is new to report - the website service will not accept any PFX exported using AES256-SHA256 encryption, it starts but the site does not respond. Re-export with the older TripleDES-SHA1 and it’s fine. Both have password set.
Not a big problem, I just have the habit now of trying to use SHA256 every time.
I’m using version: Release: 2.1.0.119 (Canary) 2025-05-29
In today’s backup, the following alert appeared:
Warnings 987
2025-06-06 23:10:08 -03 - [Warning-Duplicati.Library.Main.Operation.Backup.MetadataGenerator.Metadata-MetadataProcessFailed]: Failed to process metadata on “/Volumes/exFAT_macOS/macOS/[ Whatsapp ]/._WhatsApp Image 2024-09-12 at 09.41.10.jpeg”, storing empty metadata
FileAccesException: Unable to access the file “/Volumes/exFAT_macOS/macOS/[ Whatsapp ]/._WhatsApp Image 2024-09-12 at 09.41.10.jpeg” with method listxattr, error: EPERM (1)
And so on…
There are countless files in this and other paths, all with names starting with: ._ (dot and underscore).
They are visible or invisible files that do not exist in the paths.
Please: Any tips on how to solve this?
Thanks.
What about the previous backup? Did it work? Did anything change since it worked?
I don’t have a Mac (but I think the lead dev does), but Google can’t locate anything like “Volumes/exFAT_macOS”, so I’ll have to ask what this is that you’re wanting to backup.
Please describe the path from the left end until you can’t anymore. Maybe start from the right end and say whether there should be any .jpeg files around with names like that.
If this is actually an exFAT formatted drive, extended file attributes might not be available.
EDIT 1:
While doing Google search for “exfat” “extended attributes”, this popped up, and might fit:
Which file systems and Cloud services preserve extended attributes?
named by prefixing the main file name with ._ (period/stop, underscore)
and you wrote
EDIT 2:
AppleSingle and AppleDouble formats (Wikipedia) has more information on your files, however you should probably think through (and describe) what you have, and how it should be backed up in order to get what you want backed up. Maybe not all its files?
Although it might not work, and it might have unwanted effects, this option might help:
--skip-metadata (Boolean): Do not store metadata
Use this option to disable the storage of metadata, such as file timestamps.
I’d guess that would prevent extended attribute backup, but will it affect regular ones?
@ts678
Duplicati finds the folder correctly and backs up other items that exist in the folder (/Volumes/exFAT_macOS/macOS/[ Whatsapp ]/), but it does not back up some files (that do not exist in the folder) that start with ._
Example: ._WhatsApp Image 2024-09-12 at 09.41.10.jpeg this file does not exist in the folder. Exist file WhatsApp Image 2024-09-12 at 09.41.10.jpeg and the Duplicati backs up 100%.
EDIT: Please wait… as it is only happening in some folders, I am checking the health of the SSD. As soon as I finish the analysis, I will inform you. But it’s strange that the alerts appear in version 119 and don’t appear in version 118.
Please read my postings on these files which are seemingly “AppleDouble” to hold file information for the main file when the filesystem can’t. What is filesystem? Is it exFAT?
No such alert has been posted. Do you have one that actually says that? Your post is about metadata access by listxattr which gets extended attributes for a file that presumably exists.
The problem might be that the dot-underscore version is there for extended attributes of the base file, but then there’s nothing for the extended attributes of the dot-underscore file itself.
You could probably use a filename filter to keep Duplicati from backing up the metadata file because Duplicati would already have gotten its metadata when backing up the primary file.
This thinking should be checked by someone with macOS and Duplicati expertise (not me).
Yes, it is exFAT.
._ are macOS metadata.
In versions 118 and 119 I marked not to copy:
Hidden files
System files
Temporary files
I get the impression that in 118 these items are being excluded from the backup and in 119 they are not.
Sorry if I’m not responding properly to you.
I have a bit of difficulty with the language.
EDIT:
Due to the time clock, I’m going to close for today.
The impression I get is that:
v.118 is excluding the .hidden_file and ._file_metadata files from the backup and v.119 only the .hidden_file files.
But I don’t have enough knowledge to say for sure.
Tks and nite.
Further testing might be in order. Logs can also help. I think a verbose log will be specific:
Log may be by live log during backup, or options log-file and log-file-log-level.
I don’t see anything in the changelog indicating an intended change, and nothing in the code that’s an obviously reason for a change in 2.1.0.119, but developer may have idea (probably also much more familiar with macOS than I am) when a comment is possible.
Ok.
CONCLUSION:
Using a command in macOS Terminal, I deleted the ._ metadata files.
Now, the backup is being performed without warnings.
However, before, in v.118, the warning did not appear. Only in v.119 appear. This metadata is created when exFAT is accessed by other operating systems, for example, Windows, but version v.118 did not include the ._ metadata when making backups.
Thank you for your attention.
EDIT: Once the metadata is created again the alerts will appear again.
Tks a lot! @ts678 
You’re moving this SSD to another operating system? That might need more details.
I don’t follow the idea of another operating system being able to write Apple formats.
AppleSingle and AppleDouble formats seems to clearly describe it as macOS doing. Possibly it also involves some Windows use then a move back to macOS. Describe?
This is more for the benefit of the dev to maybe try a repro. I don’t have a Mac to test. Supplying exact steps may help. Right now, SSD moves, and “accessed” aren’t clear.
I haven’t followed the entire discussion, but here’s where things stand according to my observations over the last few days:
The TEST option or a backup run with full-remote-verification = ListAndIndexes/IndexesOnly/IndexOnly and with replace-faultiy-index-files = true and finally with no-local-db always results in errors. One or two “.dlist.zip” files are checked, never more.
Deleting and rebuilding the local database works. I have also deleted all *.dlist.zip files from the storage destination, as they are uploaded from local storage. There are hundreds of files.
The error reappears with different jobs, and sometimes it does not:
Found 1 faulty index files, repairing now” and ‘Verified 2 remote files with 1 problem(s)’.
I cannot understand why only one or two .list.zip files are always checked during the verification.
The format is exFAT (it is read/written by macOS and Windows).
I do not move the SSD.
My configuration is Dual Boot: macOS and Windows.
I use Duplicati only on macOS.
I reiterate: before (v.118), the ._ metadata already existed and the alerts did not appear.
I agree with your comment. I will wait.
Please clarify what you mean by old version. 2.1.0.119 is the current Canary version.
By problems, do you mean replace-faulty-index-files? That’s new in 2.1.0.119, therefore by definition any problems with it are new. Did anything break without that?
This used to be dangerous. A repair could upload them from database, but without a DB, deletions like that means no more backup. I’m not sure what new code does (if anything) regarding dlist file upload. Or do you mean you ran a repair and thereby got that upload?
The usual cause is that you didn’t ask for more. How many did you ask for on the test?
refers to
Duplicati.CommandLine.exe test <storage-URL> <samples> [<options>]Verifies integrity of a backup. A random sample of dlist, dindex, dblock files is downloaded, decrypted and the content is checked against recorded size values and data hashes.
<samples>specifies the number of samples to be tested. If “all” is specified, all files in the backup will be tested.
I’m coming from Canary .118. With 119, the messages appeared (more or less frequently for most jobs):
[Warning-Duplicati.Library.Main.Operation.TestHandler-FaultyIndexFiles]: Found 1 faulty index files, repairing now
[Error-Duplicati.Library.Main.Operation.TestHandler-Test results]: Verified 2 remote files with 1 problem(s)
I am aware of the risks.
Since problems with Duplicati have been occurring for years, I perform these steps from time to time (delete the local database and recreate it, delete *.dlist.zip from the storage destination).
I tried this last time because the above message appears more or less frequently for most jobs. Since .119.
That may be the point. Can I also perform a complete test in the GUI?
full-remote-verification = ListAndIndexes/IndexesOnly/IndexOnly/True does not seem to do this, regardless of which option is selected.
I think I would have to check all index files. Or regenerate them, if possible.
When deleting the *.dlist.zip, I assume that these files are only regenerated from the local database.
backup-test-percentage = 100
Only two index files are checked.
They didn’t exist before 2.1.0.119, because the option wasn’t there.
C:\Duplicati\duplicati-2.1.0.118_canary_2025-05-12-win-x64-gui>Duplicati.CommandLine help replace-faulty-index-files
Topic not found: replace-faulty-index-files
That’s unless you mean messages without it, but you posted its use.
Technically this message looks like it would be a new 2.1.0.119 one:
I’m not sure about the other because details of problem aren’t shown:
Using the Command line tools from within the Graphical User Interface
Depending on what you mean by “complete”, you probably want “all” in the Commandline arguments to test all the sample sets. You misunderstand full-remote-verification.
--full-remote-verification (Enumeration): Activate in-depth verification of files
After a backup is completed, some (dblock, dindex, dlist) files from the remote
backend are selected for verification. Use this option to turn on full
verification, which will decrypt the files and examine the insides of each
volume, instead of simply verifying the external hash. If the option
--no-backend-verification is set, no remote files are verified. This option is
automatically set when then verification is performed directly. ListAndIndexes is
like True but only dlist and index volumes are handled.
* values: True, False, ListAndIndexes, IndexesOnly, IndexOnly
* default value: False
so this is a deeper inspection of the sampled files, but it doesn’t adjust the sample size.
What adds some confusion is that it does adjust what file types are taken from sample.
Yes. That’s the risk we were talking about.
Found:
Old WebGUI → Select job → Command line → Command “TEST” → Command line arguments: make empty, enter “All”
full-remote-verification = ListAndIndexes
repalce-faultiy-index-files = true
Unfortunately I have several jobs where the errors are not fixed. So lazy index files are found and re-uploaded, but it stays that way, even after several runs.
I’ll have to have a look next weekend.
@ts678
f.y.i.
Remember? I deleted all the ._ metadata!
It turns out that every time I open a file on the SSD (exFAT format), macOS creates a ._ metadata.
So during the backup, Duplicati backs up the other files, but in the case of the ._ files, Duplicati displays the warning messages.
Again, since Duplicati v.118 does not display the warning messages, I conclude that Duplicati v.119 is handling the ._ metadata differently than Duplicati v.118.
Note: The ._ metadata have always existed in the folder in question and only now, after updating to Duplicati v.119, the alarms started to occur. I haven’t tested it, but the same problem should occur with . hidden files.
was listed under ngclient, however the bigger question is did options get moved or killed?
Options like http-operation-timeout and allowed-ssl-versions are gone in the help.
We should use a common module that defines the possible arguments, so each backend can pick the options it supports. Each backend can then report the options it supports without needing to expose all options.
but if this was done, it seems to not be working. Any user impact should be clearly noted, however ideally it would still work without requiring any migration work for previous setup.
is probably not critical to my backup, but I think some people are still needing such options.
