Error on backup to S3

Support
1

Hello!

Just installed Duplicati and let my backup run to AWS S3 Glacier Class.

The first backup ran for about 17hours (almost 400gb data/73k files) and gave three errors. A second run gave the same errors. How to I start troubleshooting what is wrong? The errors appear to be the same in the two jobs.

  • 2020-08-21 01:59:28 -04 - [Error-Duplicati.Library.Main.Operation.TestHandler-FailedToProcessFile]: Failed to process file duplicati-20200820T124355Z.dlist.zip.aes
  • 2020-08-21 02:00:10 -04 - [Error-Duplicati.Library.Main.Operation.TestHandler-FailedToProcessFile]: Failed to process file duplicati-if28efeeecc3d4d3fb425e128901e90b0.dindex.zip.aes
  • 2020-08-21 02:00:52 -04 - [Error-Duplicati.Library.Main.Operation.TestHandler-FailedToProcessFile]: Failed to process file duplicati-b5d9a6cb011e740f597ff16b28f34bbad.dblock.zip.aes
2

Welcome to the forum!

Glacier is not compatible with the testing phase of Duplicati (where it re-downloads a sample of files to check their integrity). This is because Glacier is archive tier storage and doesn’t allow immediate retrieval of files.

If you really want to use archive tier, at a minimum you will need to set a couple options: –no-backend-verification and –no-auto-compact

But I’m not really sure how Glacier complicates regular restores. Please search the forum for the term “Glacier” as there have been several discussions.

If you are open to alternatives to S3, you could consider B2 or Wasabi. Both of those offer hot storage at about the same price as AWS S3 Glacier and don’t have these issues.

3

Thank you, those options worked. No errors now.

I’ll look at B2 or Wasabi, thank you.

4

Personally I use B2. They have a service where they can ship the entire contents of a bucket to you on USB, which I thought might come in handy.

Wasabi has no egress fees which is pretty nice too. Also it is much more S3-like and can use the same type of security policies as AWS.

Good luck!