Release: 2.0.4.30 (canary) 2019-09-20

kenkendk · September 20, 2019, 7:41pm

2.0.4.30-2.0.4.30_canary_2019-09-20

Updated builds scripts to automatically notarize all packages for MacOS
Updated all string using … to us the ellipsis character, thanks @leela52452
Code quality improvements and more tests, thanks @warwickmm
Removed escaping of pipe characters on Linux, thanks @kenkendk
Updated CoCoL to avoid excessive CPU usage, thanks @kenkendk
Updated translations, thanks to all translators!
Fixed a crash when connecting to a password protected server instance from the trayicon, thanks @kenkendk

Taomyn · September 23, 2019, 5:56am

Pushed to all my Windows machines and all seems well so far

Wim_Jansen · September 24, 2019, 8:45am

Went from latest experimental to canary.

Stacktrace:

  at <unknown> <0xffffffff>
  at System.Collections.Generic.Dictionary`2<string, int>.Insert (string,int,bool) <0x00063>
  at System.Collections.Generic.Dictionary`2<string, int>.Add (string,int) <0x00027>
  at MimeKit.Utils.CharsetUtils.AddAliases (System.Collections.Generic.Dictionary`2<string, int>,int,int,string[]) <0x00077>
  at MimeKit.Utils.CharsetUtils..cctor () <0x001ef>
  at (wrapper runtime-invoke) object.runtime_invoke_void (object,intptr,intptr,intptr) <0x0006f>
  at <unknown> <0xffffffff>
  at (wrapper managed-to-native) object.__icall_wrapper_mono_generic_class_init (intptr) <0x0003b>
  at MimeKit.ParserOptions..ctor () <0x0008f>
  at MimeKit.ParserOptions..cctor () <0x00063>
  at (wrapper runtime-invoke) object.runtime_invoke_void (object,intptr,intptr,intptr) <0x0006f>
  at <unknown> <0xffffffff>
  at Duplicati.Library.Modules.Builtin.SendMail.SendMessage (string,string) <0x00117>
  at Duplicati.Library.Modules.Builtin.ReportHelper.OnFinish (object) <0x00633>
  at Duplicati.Library.Main.Controller.OnOperationComplete (object) <0x00303>
  at Duplicati.Library.Main.Controller.RunAction<T_REF> (T_REF,string[]&,Duplicati.Library.Utility.IFilter&,System.Action`1<T_REF>) <0x0054f>
  at Duplicati.Library.Main.Controller.Backup (string[],Duplicati.Library.Utility.IFilter) <0x001fb>
  at Duplicati.Server.Runner.Run (Duplicati.Server.Runner/IRunnerData,bool) <0x00e43>
  at Duplicati.Server.Program/<>c.<SetWorkerThread>b__45_0 (Duplicati.Server.Runner/IRunnerData) <0x0001b>
  at Duplicati.Library.Utility.WorkerThread`1<Tx_REF>.Runner () <0x004ab>
  at System.Threading.ThreadHelper.ThreadStart_Context (object) <0x0009f>
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) <0x001fb>
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) <0x0002b>
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object) <0x0005b>
  at System.Threading.ThreadHelper.ThreadStart () <0x0002f>
  at (wrapper runtime-invoke) object.runtime_invoke_void__this__ (object,intptr,intptr,intptr) <0x000c7>

Native stacktrace:


Debug info from gdb:


=================================================================
Got a SIGSEGV while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
=================================================================

Updated mono to the mono available at synocommunity, and now it seems to work as expected.
The crash seems to happen when sending the email, as the backup itself works OK.

If a new experimental comes out, ideally a message or warning about mono versions should be available. Ideally before activating the new release, but that might be difficult…

Regards,
Wim

kenkendk · September 24, 2019, 9:46am

Not sure exactly what goes wrong, but the stack trace indicates some character library used by MimeKit not longer performs the same way. I don’t see a good way to test this in a general manner, as it would require an extensive setup with various permutations of operating systems and library versions.

Sami_Lehtinen · September 24, 2019, 10:12am

I just tested with this version with full restore intentionally using out of sync database versus storage path. Automatic repair failed, but after manual repair everything was working again. Even the full restore was a success.

I think this starts to look good. Because the worst of my issues seems to be solved. At least based on very light testing.

Btw. --tempdir doesn’t work when restoring backup. But that’s hardly a surprise to anyone.

ts678 · September 24, 2019, 1:08pm

IMO specific failure behavior when dependencies are below spec (and spec is mono v5) is interesting to learn the symptoms (and this gave the same as After update to 2.0.4.27 duplicati crashes on synology), however a better plan is to prevent problems by ensuring dependencies are met (to the extent possible).

Every OS that Duplicati builds for probably has some way of declaring dependencies, but I don’t think the packaging and installation code has been touched. Bumped required Mono version to 3.0 for Debian build (citing a different OS build) was three years ago, however the required mono version is still 3.0 in master.

The Synology methods for declaring dependencies is in https://global.download.synology.com/download/Document/DeveloperGuide/DSM_Developer_Guide.pdf. Field Name: install_dep_packages takes version information in the INFO file but Duplicati’s doesn’t give any. There’s also PKG_DEPS also not used.

Covering dependencies for initial install doesn’t solve the autoupdate-into-fail scenario, but mono version reading at runtime appears possible. Methods range from running mono --version (but what if there are multiple mono installations – there do seem to be solutions), but one method that I tested is reflection, i.e.
GetType("Mono.Runtime" then GetMethod("GetDisplayName" as in example code in the article below:

How to find what versions of C# and .NET/Mono are being used?

Ignoring the people who had to update mono to get beyond TLS 1.0, reports so far appear to be Synology users who apparently have a large variety of mono builds due to a variety of things like CPU architectures which makes me wonder if Duplicati’s packages will look like this when it moves to bunde the framework.

One challenge with packaging is that not many people know that. One challenge with testing is access to needed equipment, but possibly @Wim_Jansen would be willing to assist, if Synology testing is needed.

Wim_Jansen · September 24, 2019, 2:11pm

I can test few things when needed on my DS216j.

kenkendk · September 25, 2019, 8:57am

That sounds like an easy fix. What files are stored outside the --tempdir folder?

Sami_Lehtinen · October 3, 2019, 8:43am

Basically everything (sqlite-temp files & dup- files etc), when restoring from storage and it does the database rebuild before actual restore. I guess that’s just one place where it’s forgotten to take it into account.

drwtsn32 · October 4, 2019, 2:19am

You are talking about “Direct restore from backup files”? Where did you define --tempdir? If it was a job-specific configuration, then that may be the issue. Direct restore from backup files won’t use any options you defined in the job configuration (unless you set them again during the restore).

If you set --tempdir at the main Duplicati settings level then I would think it should be taken into account for a direct restore.

Also, what temp folder is used incorrectly? The one pointed to by the %temp% environment variable?

Sami_Lehtinen · October 4, 2019, 8:43am

Yes, technically it’s just restore. I don’t happen to have the database, when doing disaster recovery restore (of course) as discussed earlier. With --no-local-db=true

Using --tempdir="thepath" on command-line with Duplicati.CommandLine.exe

Of course. But I think this problem is now out of scope. I don’t generally use the configuration file for stuff like this. Updating Duplicati’s own configuration files with is slightly harder with Puppet than just managing the scripts which launch the backup.

That’s exactly why I found it surprising that it didn’t work.

System default. TEMP or TMP I don’t know which one, because those both paths point to same directory in this case.

drwtsn32 · October 4, 2019, 1:19pm

Ah, if you are using Duplicati.CommandLine.exe then it doesn’t surprise me that it doesn’t use the global Duplicati setting (set in the web UI). The Command Line version is completely independent of the normal Duplicati engine and doesn’t use the main configuration settings at all.

That being said, if you are using --tempdir with the Command Line version it should work! Must be a bug…

drwtsn32 · October 4, 2019, 5:05pm

I just ran a test and confirmed this. The --tempdir option is ignored and %temp% (default) is used instead.

I will check source tonight to see if I can spot the issue.

drwtsn32 · October 5, 2019, 1:29pm

Submitted pull request 3936 which resolves the issue in my testing.