Missing LZMA and other compress methods in new versions

This is a huge blunder!!!

Some chunks are still compressed with LZMA, which is not supported in the new version. But I’ll change the method, and that should be fine. From now on, the new chunks will be compressed using the SharpCompress method. That also seems fine.

The end result? If I’m thinking correctly, it will be a completely unusable backup. Which is a critical situation! Because if recovery is needed, it won’t be possible. The reason? Recovery won’t work with either version. The old version can’t read SharpCompress chunks, while the new version can’t read LZMA chunks.

Am I right? If so, this is an extremely dangerous situation…
Especially if someone only realizes it after many months or years.

Are you referring to Zip-with-LZMA or 7z-with-LZMA2?

The “SharpCompress method” is a not entirely precise… The SharpCompress library supports a wide range of compressions, but for Duplicati we only use the zip part, and the default compression method is Deflate, which is the standard Zip compression method.

You can choose a different compression method, including LZMA, when using the SharpCompress library. Simply set --zip-compression-method=LZMA to use LZMA compression. This has been possible for many years, and is still possible with the latest releases.

I don’t understand what “SharpCompress chunks” are?
I assume the “new version” is 2.1.0.4?

If so, the “new version” supports LZMA the same as the previous version. The difference is that it will default to using a much faster Zip library by default, which only supports LZMA. If you set either one of the options --zip-compression-method=LZMA / --zip-compression-library=SharpCompress this will trigger the use of the more capable, but slower, SharpCompress library.

There is an issue when the new Zip library encounters a file with an unsupported compression method, it will fail to read the file. The intention here was to fall back to SharpCompress, but unfortunately, this fall-back is not working great. Setting --zip-compression-library=SharpCompress fixes this.

If the premise is that you can have a backup where not all files can be opened by a single version, then yes, that would make it hard to use the backup.

One way out of such a mess would be to use the RecoveryTool and the recompress method.
If you use the tool from an older version that supports reading it, it will (hopefully) be able to write it in a format that is supported by a newer tool.

But I don’t think that this is the case right now?

1 Like

and --zip-compression-library=SharpCompress until the fall-back gets better. Discussion is at:

Release: 2.0.9.108 (Canary) 2024-10-03

which could be used for a one-time back-down to something more ordinary that new zip can use.
A question would be whether the faster code can outweigh the downside of using Deflate though.

I have made a PR that fixes transparent fallback:

1 Like