ARM-based Backup Appliance?

Installation
1

Hi all.

I am trying to put together an ARM-based backup appliance using Duplicati. The board in question is an ODroid XU4 which has a 2GHz Exynos CPU (8 cores), 2GB of RAM, Gigabit Ethernet, and USB 3.0. It runs various flavors of Linux including Arch. I have a Drobo FS and a Thecus N7700ro NAS. These will be the targets for Duplicati running on my local machines. Using the appliance, I want to back them up to JottaCloud. Can I install Duplicati on ARM-based Arch?

How much working storage does Duplicati need?

2

I have tested it on RPI2 and Synology with ARM and it works there. Have not tested with ODroid.

It needs a database that stores the hashes of all data blocks, as well as all paths. Rough calculation is 42 bytes pr. block of data (defaults to --blocksize=100kb).

3

What about the AES files? Where are those held pending transfer to the Cloud? I am attaching a 9TB RAID-5 to the Odroid to house a copy of the data.

My plan is to have the workstations backup to the NAS using a file sync tool (can Duplicati do this?) then have the Odroid back up the NAS to its local array and up to JottaCloud.

I would like the copy on the NAS to be un-encrypted for easy retrieval of individual files. The Odroid would have the encrypted backups as would JottaCloud.

The Drobo FS went to Bit Bucket Heaven so I am harvesting the drives and using them in a USB 3.0 RAID enclosure attached to the Odroid.

4

They are stored in the system temp folder until they can be uploaded. By default there will only be 2 files written in advance.

No, Duplicati does not do sync, only backup.

Not sure how this is set up, but normally Duplicati is installed on the machine hosting the files. You can work around this by mounting the NAS on the ODroid via SMB or something else, and then backup from the mount point to the disk array.

5

The temp files cannot use the system Temp folder on an Odroid because it boots from a microSD card, not a normal hard drive. Can the location of the temp folder be specified? I would rather have it on the locally attached array.

6

Yes, best way is to set TMPDIR environment variable, as SQLite will use that to place transaction files as well.

8

I have Duplicati starting, but I am getting the following error in the logs:

Sep 18 20:56:49 clouddroid mono[297]: Error message: System.TypeInitializationException: The type initializer for 'Mono.Unix.Native.NativeConvert' threw an exception. ---> System.IO.FileNotFoundException: Could not find file "/etc/localtime"
Sep 18 20:56:49 clouddroid mono[297]:   at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share, System.Int32 bufferSize, System.Boolean anonymous, System.IO.FileOptions options) [0x001d8] in <09e02dbedac243edb3014e35
Sep 18 20:56:49 clouddroid mono[297]:   at System.IO.FileStream..ctor (System.String path, System.IO.FileMode mode, System.IO.FileAccess access, System.IO.FileShare share) [0x00000] in <09e02dbedac243edb3014e35f4e0d1ad>:0
Sep 18 20:56:49 clouddroid mono[297]:   at (wrapper remoting-invoke-with-check) System.IO.FileStream:.ctor (string,System.IO.FileMode,System.IO.FileAccess,System.IO.FileShare)
Sep 18 20:56:49 clouddroid mono[297]:   at System.IO.File.OpenRead (System.String path) [0x00000] in <09e02dbedac243edb3014e35f4e0d1ad>:0
Sep 18 20:56:49 clouddroid mono[297]:   at System.TimeZoneInfo.FindSystemTimeZoneByFileName (System.String id, System.String filepath) [0x0000e] in <09e02dbedac243edb3014e35f4e0d1ad>:0
Sep 18 20:56:49 clouddroid mono[297]:   at System.TimeZoneInfo.CreateLocal () [0x000cc] in <09e02dbedac243edb3014e35f4e0d1ad>:0
Sep 18 20:56:49 clouddroid mono[297]:   at System.TimeZoneInfo.get_Local () [0x00009] in <09e02dbedac243edb3014e35f4e0d1ad>:0
Sep 18 20:56:49 clouddroid mono[297]:   at System.CurrentSystemTimeZone..ctor () [0x00006] in <09e02dbedac243edb3014e35f4e0d1ad>:0
Sep 18 20:56:49 clouddroid mono[297]:   at System.TimeZone.get_CurrentTimeZone () [0x0003c] in <09e02dbedac243edb3014e35f4e0d1ad>:0
Sep 18 20:56:49 clouddroid mono[297]:   at Mono.Unix.Native.NativeConvert..cctor () [0x00026] in <dd5b78b86c614c3ab37b4f41920bee3a>:0
Sep 18 20:56:49 clouddroid mono[297]:    --- End of inner exception stack trace ---
Sep 18 20:56:49 clouddroid mono[297]:   at Mono.Unix.Native.Syscall.lseek (System.Int32 fd, System.Int64 offset, Mono.Unix.Native.SeekFlags whence) [0x00000] in <dd5b78b86c614c3ab37b4f41920bee3a>:0
Sep 18 20:56:49 clouddroid mono[297]:   at Mono.Unix.UnixStream..ctor (System.Int32 fileDescriptor, System.Boolean ownsHandle) [0x0003b] in <dd5b78b86c614c3ab37b4f41920bee3a>:0
Sep 18 20:56:49 clouddroid mono[297]:   at Mono.Unix.UnixStream..ctor (System.Int32 fileDescriptor) [0x00000] in <dd5b78b86c614c3ab37b4f41920bee3a>:0
Sep 18 20:56:49 clouddroid mono[297]:   at (wrapper remoting-invoke-with-check) Mono.Unix.UnixStream:.ctor (int)
Sep 18 20:56:49 clouddroid mono[297]:   at UnixSupport.File.OpenExclusive (System.String path, System.IO.FileAccess mode, System.Int32 filemode) [0x0007d] in <3fbec333f978484785726c089e2a43ac>:0
Sep 18 20:56:49 clouddroid mono[297]:   at UnixSupport.File.OpenExclusive (System.String path, System.IO.FileAccess mode) [0x00000] in <3fbec333f978484785726c089e2a43ac>:0
Sep 18 20:56:49 clouddroid mono[297]:   at Duplicati.Server.SingleInstance..ctor (System.String appname, System.String basefolder) [0x001e5] in <f6ac143bad464e5896e7e3eec4e0f7b9>:0
Sep 18 20:56:49 clouddroid mono[297]:   at Duplicati.Server.Program.RealMain (System.String[] args) [0x00242] in <f6ac143bad464e5896e7e3eec4e0f7b9>:0

The web interface does not load. Netstat -l returns the following:

[guru@clouddroid ~]$ netstat -l
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 0.0.0.0:ssh             0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:microsoft-ds    0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:netbios-ssn     0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:llmnr           0.0.0.0:*               LISTEN
tcp6       0      0 [::]:ssh                [::]:*                  LISTEN
tcp6       0      0 [::]:microsoft-ds       [::]:*                  LISTEN
tcp6       0      0 [::]:llmnr              [::]:*                  LISTEN
tcp6       0      0 [::]:netbios-ssn        [::]:*                  LISTEN
udp        0      0 localhost:domain        0.0.0.0:*
udp        0      0 192.168.1.25:netbios-ns 0.0.0.0:*
udp        0      0 clouddroid:netbios-ns   0.0.0.0:*
udp        0      0 0.0.0.0:netbios-ns      0.0.0.0:*
udp        0      0 192.168.1.2:netbios-dgm 0.0.0.0:*
udp        0      0 clouddroid:netbios-dgm  0.0.0.0:*
udp        0      0 0.0.0.0:netbios-dgm     0.0.0.0:*
udp        0      0 0.0.0.0:llmnr           0.0.0.0:*
udp6       0      0 [::]:llmnr              [::]:*
Active UNIX domain sockets (only servers)
Proto RefCnt Flags       Type       State         I-Node   Path
unix  2      [ ACC ]     STREAM     LISTENING     15917    /var/run/samba/nmbd/unexpected
unix  2      [ ACC ]     STREAM     LISTENING     14410    /run/systemd/private
unix  2      [ ACC ]     SEQPACKET  LISTENING     4454     /run/udev/control
unix  2      [ ACC ]     STREAM     LISTENING     14449    /run/systemd/journal/stdout
unix  2      [ ACC ]     STREAM     LISTENING     15243    /run/user/1001/systemd/private
unix  2      [ ACC ]     STREAM     LISTENING     15248    /run/user/1001/bus
unix  2      [ ACC ]     STREAM     LISTENING     15250    /run/user/1001/gnupg/S.gpg-agent.extra
unix  2      [ ACC ]     STREAM     LISTENING     15253    /run/user/1001/gnupg/S.gpg-agent.ssh
unix  2      [ ACC ]     SEQPACKET  LISTENING     14486    /run/systemd/coredump
unix  2      [ ACC ]     STREAM     LISTENING     15255    /run/user/1001/gnupg/S.gpg-agent
unix  2      [ ACC ]     STREAM     LISTENING     15257    /run/user/1001/gnupg/S.dirmngr
unix  2      [ ACC ]     STREAM     LISTENING     15259    /run/user/1001/gnupg/S.gpg-agent.browser
unix  2      [ ACC ]     STREAM     LISTENING     15540    /run/dbus/system_bus_socket
unix  2      [ ACC ]     STREAM     LISTENING     4576     /run/lvm/lvmetad.socket

Any ideas?

9

That is a Mono issue:
https://bugzilla.xamarin.com/show_bug.cgi?id=56499

10

kenkendk, do you know if a standard mono update will fix it yet?

Also, would it be useful to include mono information in the About page since it seems there are some issues with certain versions?

Edit:
Never mind - I’m just lazy and/or blind…it’s already in the “About -> System Info” page.

System properties

APIVersion : 1
PasswordPlaceholder : **********
ServerVersion : 2.0.2.1
ServerVersionName : - 2.0.2.1_beta_2017-08-01
ServerVersionType : Beta
BaseVersionName : 2.0.2.1_beta_2017-08-01
DefaultUpdateChannel : Beta
DefaultUsageReportLevel : Information
ServerTime : 2017-09-21T13:40:55.2060941-05:00
OSType : Windows
DirectorySeparator : \
PathSeparator : ;
CaseSensitiveFilesystem : false
MonoVersion :                      <------ HERE (blank for windows)
11

Here’s a workaround:

Install tzdata (sudo pacman -S tzdata)
Edit the symlink /etc/localtime and have it point to the actual filename set up by tzdata. (Mine said “New York” when it should have been “New_York”.
Restart Duplicati and you should be golden.

13

How do I set that? I edited /etc/environment and it is still writing to /tmp.

14

I would guess something like this works:

export TMPDIR=/mytemp
duplicati-server &
15

Well I added the export to /etc/profile and it made no difference. I will ask in the Arch subreddit and see if I get an answer.

16

I did a printenv and both TMPDIR and DUPLICATI_DATA_FOLDER are there and being ignored.

17

Can you try TMP environment variable?
I found that on Windows this is the only one working, although I saw code going through all three of them (incl. TEMP).

18

Here is the output of my printenv command:

[guru@clouddroid temp]$ printenv
SSH_CONNECTION=192.168.1.147 61973 192.168.1.98 22
LANG=C
TZ=/usr/share/zoneinfo/America/New_York
XDG_SESSION_ID=c1
USER=guru
PWD=/media/raid/temp
HOME=/home/guru
TMP=/media/raid/temp
SSH_CLIENT=192.168.1.147 61973 22
DUPLICATI_DATA_FOLDER=/media/raid/databases
TMPDIR=/media/raid/temp/
SSH_TTY=/dev/pts/0
MAIL=/var/spool/mail/guru
TERM=xterm
SHELL=/bin/bash
SHLVL=1
LOGNAME=guru
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1001/bus
XDG_RUNTIME_DIR=/run/user/1001
PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl _=/usr/bin/printenv
OLDPWD=/home/guru
[guru@clouddroid temp]$

Duplicati is ignoring it and writing /tmp

19

Are both being ignored? In that case I think it has something to do with the process starting with another environment (maybe already running in ano. You can run:

duplicati-cli help --tempdir

It shows the “default value” for the option, which is the temporary folder. If you have TMPDIR set, it should show the right value.

20

OK, I got the temp files writing to the directory I wanted by adding this to the systemd/system/duplicati.service file:

[Service]
Environment=TMPDIR=/media/raid/temp

Now the temp files are written to the RAID array instead of the SD card. The sqllite files are still written to the default location /root/.config/Duplicati-server.sqlite

How do I move those?

21

You can set the environment variable Duplicati_HOME to point to the folder you want to use.

I assume you can set it like this (unsure of the syntax though):

[Service]
Environment=TMPDIR=/media/raid/temp:Duplicati_HOME=/media/raid/duplicati_storage
22

It’s on multiple Environment entries, but yes, that is how I got it to work. Thanks for all of your help.