Is 7z safe to use yet?

[kenkendk]

@prez

I saw in another post that using 7z instead of zip is unsafe. Is this still the case? or is 7z now safe to use? I would like to switch to using it doesn’t have any issues.

It has a number of issues. We need to rewrite the 7z module to use the latest LZMA2 code, but no one has prioritized doing so.

@prez

7z results in a much smaller archive, and more importantly, can be multi-threaded (which zip can not).

In our tests we found that 7z gives a minor space saving (~10%) but takes much longer to compress. With the latest canary release, Duplicati is able to run multiple zip compressions in parallel, giving many of the benefits from 7z.

The multi-threading for LZMA2 is actually the main problem with the 7z module. It works with very large buffers, making it hard to know in advance when the buffer is filled, causing Duplicati to be cautious and ending up creating many small volumes.

@prez

Additionally, is there any plans to introduce more common archive formats (for unix sytems) like gz and bz2 (both of which can also be multi-threaded and are VERY efficient and should be very well supported across platforms).

AFAIK, gz and bz2 are just compression algorithms, they do not have a file format, relying often on tar. You can get some of the same by choosing another compression algorithm for the zip module (--zip-compression-method=bzip2).

The tar file format is a mess with multiple vendor extensions and guessing to figure out which version.

@prez

BTW, right now, I’m using the beta release channel, is multi-threading only available on the canary tree?

Yes, multi-threading is still being tested to make sure it does not trip up.

@prez

And is there going to be an update TO the canary or beta tree (the last release for both is in April, and looking at the git repository, there have been patches that fix issues since then).

Yes! I had some pressing work so I had to leave Duplicati behind for a while. I have fixed the performance issues today, and hope to get a new canary out very soon.

The beta will be updated once we are confident that the canary changes do not cause issues.