rpmlib(PayloadIsZstd) <= 5.4.18-1 is needed by duplicati*.noarch.rpm

Installation
#1

Hello,

I’m running a Duplicati Client on a Win10 machine with the target system being a homebrew NAS based on Linux Opensuse Leap 15.1 as Samba Fileserver locally over LAN.

I’m VERY happy with how Duplicati is designed and it works for me perfectly.

Besides the Client installation on the Win10 machine, I have also maintained the installation of Duplicati under Opensuse. I’m using this local installation of Duplicati for integrity tests, (running “test all” and also a full restore from time to time on the linux box).

The installation of the duplicati*noarch.rpm has been successful in the past, the issue with libappindicator is also sorted, everything was fine up to the duplicati-2.0.4.38 (canary) / duplicati-2.0.5.1 (beta) respectively.

However, starting with duplicati-2.0.5.101 (canary) in January 2020 i can’t install the duplicati*noarch.rpm any more:

rpm -Uvh --nodeps duplicati-2.0.5.103-2.0.5.103_canary_20200218.noarch.rpm
Preparing...                          ################################# [100%]
Updating / installing...
   1:duplicati-2.0.5.103-2.0.5.103_can################################# [ 50%]
error: unpacking of archive failed: cpio: Bad magic
error: duplicati-2.0.5.103-2.0.5.103_canary_20200218.noarch: install failed

rpm -Uvh --test duplicati-2.0.5.103-2.0.5.103_canary_20200218.noarch.rpm
error: Failed dependencies:
        libappindicator is needed by duplicati-2.0.5.103-2.0.5.103_canary_20200218.noarch
        rpmlib(PayloadIsZstd) <= 5.4.18-1 is needed by duplicati-2.0.5.103-2.0.5.103_canary_20200218.noarch

The “Bad Magic” seems to be caused by the issue of the “PayloadIsZstd”.

I have tried to educate me about this rpmlib and Zstd, but there is not much info out there or I have missed it. I appears to me that the rpm package is now compressed with a different method that is not supported by the rpm of Opensuse Leap 15.1. However, I might be wrong.

Also, is it only me having the isssue?
Can anyone give a hint how to resolve this issue, please?

Thanks in advance!
Paul

Upgrade from beta to canary on CentOS
#2

Hello, welcome to the forum!

A change was made less than a month ago to the rpm package dependencies. Intention was to require mono 5.x or higher. Not sure if it is related to your issue but the timing is suspicious.

What version of mono is on this system?

#3

Thank you very much for the kind welcome!

The system is Opensuse Leap 15.1 with a current patch level.
mono-core package version is 5.10.1-lp151.2.59
rpm package version is 4.14.1-lp151.13.0

The old duplicati versions up to 2.0.51 still install just fine.

Paul

#4

Ok, I’m going to try and reproduce your problem… stay tuned…

#5

Wow, … thank you!

#6

The rpm compression has changed from xz to zstd for some reason. Maybe that causes the issue?

$ rpm -qp --qf '%{PAYLOADCOMPRESSOR}\n' duplicati-2.0.5.1-2.0.5.1_beta_20200118.noarch.rpm
xz
$ rpm -qp --qf '%{PAYLOADCOMPRESSOR}\n' duplicati-2.0.5.101-2.0.5.101_canary_20200123.noarch.rpm
zstd
$

Changes/Switch RPMs to zstd compression from Fedora.

[RFC] Changes/Switch RPMs to zstd compression #11 from openSUSE about dealing with this change.

Bug 1715799 - Re-enable zstd compression support in RHEL-8 rpm Red Hat Enterprise Linux 8 on this.

I assume Duplicati is more interested in a universal install rather than speed. Would forcing xz get that?
Sorry I don’t know much about how the Fedora build works. Might need to get in an expert or several…

Meanwhile, there’s a chance rpm2cpio can convert the file, if you want to see whether it runs if installed.

#7

Sounds like a few distributions are moving towards zstd for packaging. Arch has done so as well.

#8

But can a distro that uses zstd as its own standard generally also install older standards such as xz? Duplicati presumably doesn’t want to do multiple builds, but an old standard may work on new distro. There are also lots of old distros installed, and Duplicati presumably wants to fit as many as possible.
What surprises me, if this is a compression mismatch, is that it’s not been found before. Few installs?
Possibly there’s an alternate theory, but I thought I’d put this one out as a possible one to start with…

#9

Yep that looks like the problem. I was able to reproduce the issue. OpenSUSE 15.1 lacks the library needed to extract the files from the RPM.

#10

Thank you for the hint, but it seems that rpm2cpio on opensuse leap 15.1 can’t handle zstd neither.

rpm2cpio duplicati-2.0.5.101-2.0.5.101_canary_20200123.noarch.rpm | cpio -t
spits out just some garbage for duplicati versions 2.0.5.101 and up. Works nicely for older versions, though.

#11

Hello, I seem to be having the similar problem on CentOS 7.5
I installed zstd and tried rpm -Va --nofiles --nodigest but no love.

Dependencies Resolved

============================================================================================================================================
Package Arch Version Repository Size

Installing:
duplicati noarch 2.0.5.103-2.0.5.103_canary_20200218 /duplicati-2.0.5.103-2.0.5.103_canary_20200218.noarch 32 M
Installing for dependencies:
atk x86_64 2.28.1-1.el7 base 263 k
desktop-file-utils x86_64 0.23-2.el7 base 67 k
emacs-filesystem noarch 1:24.3-22.el7 base 58 k
fribidi x86_64 1.0.2-1.el7_7.1 updates 79 k
gdk-pixbuf2 x86_64 2.36.12-3.el7 base 570 k
glib-sharp2 x86_64 2.12.45-0.xamarin.7.epel7 download.mono-project.com_repo_centos7_ 70 k
graphite2 x86_64 1.3.10-1.el7_3 base 115 k
gtk-sharp2 x86_64 2.12.45-0.xamarin.7.epel7 download.mono-project.com_repo_centos7_ 779 k
gtk-update-icon-cache x86_64 3.22.30-3.el7 base 28 k
gtk2 x86_64 2.24.31-1.el7 base 3.4 M
harfbuzz x86_64 1.7.5-2.el7 base 267 k
hicolor-icon-theme noarch 0.12-7.el7 base 42 k
jasper-libs x86_64 1.900.1-33.el7 base 150 k
libXcomposite x86_64 0.4.4-4.1.el7 base 22 k
libXcursor x86_64 1.1.15-1.el7 base 30 k
libXft x86_64 2.3.2-2.el7 base 58 k
libXi x86_64 1.7.9-1.el7 base 40 k
libXinerama x86_64 1.1.3-2.1.el7 base 14 k
libXrandr x86_64 1.5.1-2.el7 base 27 k
libappindicator x86_64 12.10.0-13.el7 base 37 k
libappindicator-sharp x86_64 12.10.0-12.el7 epel 27 k
libdbusmenu x86_64 16.04.0-4.el7 base 132 k
libdbusmenu-gtk2 x86_64 16.04.0-4.el7 base 34 k
libindicator x86_64 12.10.1-6.el7 base 63 k
libthai x86_64 0.1.14-9.el7 base 187 k
pango x86_64 1.42.4-4.el7_7 updates 280 k

Transaction Summary

Install 1 Package (+26 Dependent packages)

Total size: 38 M
Installed size: 54 M
Is this ok [y/d/N]: y
Downloading packages:
Running transaction check
ERROR You need to update rpm to handle:
rpmlib(PayloadIsZstd) <= 5.4.18-1 is needed by duplicati-2.0.5.103-2.0.5.103_canary_20200218.noarch
RPM needs to be updated
You could try running: rpm -Va --nofiles --nodigest
Your transaction was saved, rerun it with:
yum load-transaction /tmp/yum_save_tx.2020-03-13.19-30.cAPk7D.yumtx
[root@s192-169-155-147 sourcefiles]# rpm -Va --nofiles --nodigest duplicati-2.0.5.103-2.0.5.103_canary_20200218.noarch

#12

Today I updated my system from Linux Opensuse Leap 15.1 to Opensuse Leap 15.2 and the installation issue described in my original post is gone. Seems that the rpm tool installed with the latest Opensuse version is now capable of understanding the new “Zstd” format.

I was able to update the Duplicati rpm to 2.0.5.106 and then to 2.0.5.107. Very nice, indeed.
Unfortunately the latest release for 2.0.5.108 is missing the rpm package, but this is already another ticket.

1 Like
#13

This problem persists in CentOS 7.8 as well.

The CentOS RPM lib does not support the Zstd compression.

If the RPMs were build using a different compression it should work.

#14

Could someone please file a GitHub issue? We need to find someone who knows why/how build changed. Linking to this issue, and also to Upgrade from beta to canary on CentOS would add some context for that.
Presumably the people reporting this are on Canary 2.0.5.101 or later (thanks for helping as early warning).

#15

It looks like the rpm is built using rpmbuild from the fedora:latest Docker image, which apparently was changed to use zstd in Fedora 31. According to this, the latest tag was changed to Fedora 31 in late 2019, so I’m not sure how to reconcile the timing with Duplicati 2.0.5.101.

#16

If the latest Fedora is grabbed each build, that seems prone to unexpected uncontrolled breakages.
If the build server is updated manually, then the Oct 28, 2019 change you cited could be taken later.

I wonder if build changes caused 2.0.5.100_canary_2020-01-18 to release just source, no binaries?
But the Duplicati build is not an area I know, so I’m hoping someone who knows build can be found.

I’m a little surprised that having a single rpm for all rpm-consuming Linux seems fairly lib-issue-free.
Compression issue might be solvable by using Fedora 30 if the current build is using Fedora 31, or:

How do you configure rpm-build on fedora 31 to use xz and NOT zstd? has an inconclusive method.
Never use ‘xz’ compression cuz like, older centos doesn’t support it #192 is pain of previous switch
Tell rpm to use gzip for compression and md5 for checksums was the first fix. Fancier one followed.
Add --rpm-digest and --rpm-compression flags to allow selectable might map compression macros:

    "xz" => "w2.xzdio",
    "gzip" => "w9.gzdio",

Features/XZRpmPayloads

In order to turn on XZ payload compression for binary RPMS one could define following macros in /usr/lib/rpm/macros globally or in ~/.rpmmacros file locally:
%_binary_payload w7.xzdio

This enables XZ payload compression level 7 (suggested default by upstream) for binary RPMs. This macro isn’t defined by default and if undefined gzip compression is used.

The %_source_payload macro can be used to change the payload compression for source RPMs.

Changes/Switch RPMs to zstd compression

The macro for setting the compression is: %define _binary_payload w19.zstdio
SRPM payload compression should stay at gzip (there’s almost no benefit in changing the compression, because SRPM’s contents is compressed already)

The comparison table compares xz level 2 against zstd level 19. I’m not sure what xz level Fedora 30 actually wound up doing, but if more web searches can’t answer, a look at Fedora 30 might be able to.

#17

bug report filed

#18

Is the duplicati source RPM or the spec file available for download somewhere? That would enable everyone to build the rpm on their own system.

#19

https://www.duplicati.com/download links to https://github.com/duplicati/duplicati/releases

https://github.com/duplicati/duplicati/tree/master/Installer/fedora if you only want spec files.