Running on MacOS Catalina Beta

Duplicati 2.0.4.19_canary_2019-06-17 is working fine on MacOS Catalina Beta with some minor tweaks. Two tweaks previously documented on this forum and a new third tweak. I could not get 2.0.4.5_beta_2018-11-28 to work reliably so skip straight to Canary.

First step is to run duplicati-server as a LaunchDaemon and the tray-icon as a LaunchAgent as documented here. Copy each plist into /Library/LaunchDaemons and /Library/LaunchAgents then load the plists into launchctl.

launchctl load -w /Library/LaunchAgents/net.duplicati.tray-icon.plist
launchctl load -w /Library/LaunchDaemons/net.duplicati.server.plist

Second step is to grant Full Disk Access to /Applications/Duplicati.app/ under System Preferences ā†’ Security & Privacy ā†’ Privacy as documented here. When youā€™re done it should look like this.

Third step is to grant Full Disk Access to /bin/bash. Be warned, this is insecure and is a quick and dirty workaround. Iā€™m doing this on a personal machine but Iā€™d think twice before doing this on a business laptop. As in the previous step click the + button then Cmd-Shift-G then enter /bin/bash. When youā€™re done it should look like this.

Discussing that last step. Mojave puts a sandbox around sensitive files such as Photos and Mail as an additional security layer on top of UNIX file permissions. The access should apply to all contents of /Applications/Duplicati.app/ including duplicati-server. In Mojave this worked. In Catalina itā€™s broken. You see access errors in the logs same as if Duplicati.app hadnā€™t been granted Full Disk Access.

[Warning-Duplicati.Library.Main.Operation.Backup.FileEnumerationProcess-FileAccessError]: Error reported while accessing file: /Users/nhand42/Pictures/Photos Library.photoslibrary/

I tried granting Full Disk Access to /Library/Frameworks/Mono.framework/Versions/5.18.1/bin/mono-sgen64 and /Applications/Duplicati.app/Contents/MacOS/duplicati-server but neither of those worked. But after granting access to /bin/bash it started backing up Photos and Mail. My best guess is Catalina doesnā€™t consider /bin/bash to be part of the /Applications/Duplicati.app/ bundle so the trust isnā€™t inherited.

An experiment would be to copy all dependencies including bash and mono under Duplicati.app/Contents/MacOS and modifying all the scripts to use the bundled versions.

2 Likes

Hi @nhand42, welcome to the forum!

Thanks for sharing your workaround for MacOS Catalina beta. I agree that bash permission tweak feels pretty ugly. Unfortunately I donā€™t have a machine that can run Catalina. :frowning:

Iā€™m curious, what about 2.0.4.5 beta was unreliable for you?

Iā€™m curious, what about 2.0.4.5 beta was unreliable for you?

Show Logs wasnā€™t working. It would only show a blank page. It also reported a corrupt database shortly after the Catalina upgrade. I started a repair but by that stage Iā€™d lost faith so blew it away and started with a clean install of Canary.

Iā€™m confident the Catalina upgrade was the source of both problems. Apple has made some sweeping changes to the filesystem which has broken other file-based software including Microsoft OneDrive. A developer has documented the changes which Iā€™ve linked below.

Unfortunately I donā€™t have a machine that can run Catalina.

Iā€™m running a second instance of Catalina using kvm on a bog standard Debian 9.0 installation. I can share the details if that would help for testing.

Thanks for this! Fixed it perfectly. I didnā€™t bother running the server and client separately, just granted full disk access to Duplicati.app and bash.

Considering macOS 10.15 Catlina has officially released, is macOS unsupported now with Duplicati?

Thanks for posting the bash workaround. After upgrading to Catalina and working through the mono/cert issue (again), this was also needed to get Duplicati access to a USB removable drive in a backup source. (Iā€™m on 2.0.4.23_beta.)

Any plan to patch duplicati so users do not have to use this workaround? I am hesitant to upgrade to Catalinaā€¦

I tried to use this work around to be able to backup photos and mail. After granting full disk access to bash I get the following error when trying to run a backup-
Failed: The database file is locked
database is locked
Details: Mono.Data.Sqlite.SqliteException (0x80004005): The database file is locked
database is locked
at Mono.Data.Sqlite.SQLite3.Step (Mono.Data.Sqlite.SqliteStatement stmt) [0x00089] in <6ebdbeb8eb164ecf9c9f787f8ac2dca0>:0
at Mono.Data.Sqlite.SqliteDataReader.NextResult () [0x00104] in <6ebdbeb8eb164ecf9c9f787f8ac2dca0>:0
at Mono.Data.Sqlite.SqliteDataReaderā€¦ctor (Mono.Data.Sqlite.SqliteCommand cmd, System.Data.CommandBehavior behave) [0x0004e] in <6ebdbeb8eb164ecf9c9f787f8ac2dca0>:0
at (wrapper remoting-invoke-with-check) Mono.Data.Sqlite.SqliteDataReader:.ctor (Mono.Data.Sqlite.SqliteCommand,System.Data.CommandBehavior)
at Mono.Data.Sqlite.SqliteCommand.ExecuteReader (System.Data.CommandBehavior behavior) [0x00006] in <6ebdbeb8eb164ecf9c9f787f8ac2dca0>:0
at Mono.Data.Sqlite.SqliteCommand.ExecuteDbDataReader (System.Data.CommandBehavior behavior) [0x00000] in <6ebdbeb8eb164ecf9c9f787f8ac2dca0>:0
at System.Data.Common.DbCommand.System.Data.IDbCommand.ExecuteReader () [0x00000] in <615d30664e0f432b92ce8b521d3f9739>:0
at Duplicati.Library.Main.Database.ExtensionMethods.ExecuteScalarInt64 (System.Data.IDbCommand self, System.Boolean writeLog, System.String cmd, System.Int64 defaultvalue, System.Object values) [0x00061] in :0
at Duplicati.Library.Main.Database.ExtensionMethods.ExecuteScalarInt64 (System.Data.IDbCommand self, System.String cmd, System.Int64 defaultvalue, System.Object values) [0x00000] in :0
at Duplicati.Library.Main.Database.LocalDatabaseā€¦ctor (System.Data.IDbConnection connection, System.String operation) [0x0005e] in :0
at Duplicati.Library.Main.Database.LocalDatabaseā€¦ctor (System.String path, System.String operation, System.Boolean shouldclose) [0x00007] in :0
at Duplicati.Library.Main.Database.LocalBackupDatabaseā€¦ctor (System.String path, Duplicati.Library.Main.Options options) [0x00000] in :0
at Duplicati.Library.Main.Operation.BackupHandler+d__19.MoveNext () [0x00042] in :0
ā€” End of stack trace from previous location where exception was thrown ā€”
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in :0
at CoCoL.ChannelExtensions.WaitForTaskOrThrow (System.Threading.Tasks.Task task) [0x00050] in <6973ce2780de4b28aaa2c5ffc59993b1>:0
at Duplicati.Library.Main.Operation.BackupHandler.Run (System.String sources, Duplicati.Library.Utility.IFilter filter) [0x00008] in :0
at Duplicati.Library.Main.Controller+<>c__DisplayClass13_0.b__0 (Duplicati.Library.Main.BackupResults result) [0x00035] in :0
at Duplicati.Library.Main.Controller.RunAction[T] (T result, System.String& paths, Duplicati.Library.Utility.IFilter& filter, System.Action`1[T] method) [0x0011d] in :0

No idea what I did wrong?

Nevermind.:roll_eyes:
Looks like a reboot has fixed the problem.

1 Like

It sounds strange to me that the logs page would stop working. It is just a list of messages from the server (either from memory or the database). I cannot think of anything the OS could have changed to break that feature.

The corrupt database also sounds weird. Duplicati has some known issues where it breaks the database, which might be what you are seeing. If the database is corrupt (as in cannot open with SQLite browser), then it might be an OS issue.

No, I still consider it working. The increased security settings are problematic, because Duplicati needs access to many of the items considered sensitive (images, email, etc). I hope we can find a workaround to avoid the need for giving /bin/bash access.

It could be caused by zsh now being the default shell. The Duplicati launcher uses bash to start the process, but we could switch to zsh if that removes the need for giving the shell full access:
https://github.com/duplicati/duplicati/blob/master/Installer/OSX/Duplicati-trayicon-launcher

Hello!

Apparently, this is still an issue with version 2.0.4.23_beta_2019-07-14.
Iā€™m on the Catalina release and I canā€™t access my Documents folder, even after giving duplicati and bash full disk access.
I wanā€™t really sure about the instruction under ā€˜step oneā€™, since the version was a bit older and the explanation was a bit sketchy for me.

Just when i thought I found a backup solution for all my OSs. Welcome to the joy of cross plattform :wink:

If someone could elaborate on this first step and/or the current state of affairs, that would be very helpful.

Thank you!

1 Like

I think you need to restart Duplicati for the permissions to kick in. I have not tested this, but if you are using an updated version that runs from the update folder, it is possible that it does not inherit the permissions.

The first step just registers Duplicati to run as a system service, it is not required for normal operations.

I tried various workarounds for this, but could not get it to work. If I grant terminal full access, I can launch the app from the commandline and all works. But no matter how I try to get it to start the scripts, it drops the privileges. It looks like it happens by design, and I have not managed to work around it.

Since Python will no longer be bundled with MacOS in the future, I think the only option is to write a launcher in ObjC or similar, that replicates the mono locator and starts it. It should be easy to test if this approach works without adding the full functionality from run-with-mono.sh.

EDIT: This approach works. If the launcher is an MacOS executable, it correctly obtains permissions. You also get the normal ā€œdo you want to give this program access?ā€ dialogs if you do not give it full permissions.

Here is a PR that makes native launchers, and in my tests this works with Gatekeeper in macOS Catalina requesting permission, and also accepting if you give Duplicati ā€œfull disk accessā€:

2 Likes

Did anyone manage to try the latest canary build? It contains brand new launchers that do not depend on Bash. On my machine this works nicely with Gatekeeper and requests access whenever Duplicati accesses a protected folder. It also works if I grant full access to Duplicati.app.

If anyone has tried the canary with Catalina, could you report back on success/failure ?

Iā€™m running the latest 2.0.4.33_canary_2019-11-01 with the new launchers for the past week. Works perfectly now. Iā€™ve also removed Full Disk Access for /bin/bash. Thanks for fixing this.

1 Like

It works for me tooā€”I didnā€™t grant Full Disk Access to /bin/bash in the first place, but the 2.0.4.33_canary_2019-11-01 build I have works perfectly. I did grant Full Disk Access to Duplicati earlier, and I havenā€™t tested running without that.

1 Like

I had this issue, and 2.0.4.38_canary_2019-12-29 solved it with full disk access granted to Duplicati on Catalina (not bin/bash, thankfully, which was deeply scary). Many thanks.

1 Like