Run-script-after not work

xblitz · August 2, 2019, 2:33pm

Hi!
I’m a new user of Duplicati; I start use this software same months ago and continue to use it! In these months I encountered some problems (very well know), anyway I belive in this project but unfortunately in Italy this software isn’t very well known.

Now, this is my problem: I wish backup the local database so if my HDD suddenly death I should be able to recovery my data without waiting hours or days to recreate the whole databases from scratch. I create a batch file and use the --run-script-after option to launch it when the backup has finished. In the script I launch duplicati client to schedule the backup of databases when the last backup job finish successifully. Unfortunately the backup finish correctly but it exit with a warning says “Esecuzione dello script ” fuori tempo" (in english) “Execution of the script out of time”.

Of course if I run the script from cmd.exe it works fine. If I use the option --run-script-timeout I obtain the same result. If I use this dummy script I obtain the same result

Blockquote
@echo off
if /i “%DUPLICATI__OPERATIONNAME%” neq “Backup” goto end
if /i “%DUPLICATI__PARSED_RESULT%” neq “Success” goto end
c:\Windows\System32\notepad.exe
:end
exit 0

If I replace notepad.exe with a echo It works correctly. Can anyone help me?

2.0.4.22_canary_2019-06-30 on windows 10
Thanks

drwtsn32 · August 2, 2019, 4:37pm

In your dummy script you are running notepad.exe. This will pause the script until notepad.exe is closed, leading to the timeout error.

In your real script you aren’t running notepad.exe, but what command are you running? Can you share your real script? (Remove sensitive information if present…)

xblitz · August 2, 2019, 4:46pm

Ouch! It’s true: notepad.exe is a bad executable for my experiment. I must repeat my test. Anyway this is my true script

@echo off
if /i “%DUPLICATI__OPERATIONNAME%” neq “Backup” goto end
if /i “%DUPLICATI__PARSED_RESULT%” neq “Success” goto end
rem timeout /t 5 /nobreak > nul
D:\duplicati_aux\duplicati_client.exe login
rem timeout /t 5 /nobreak > nul
D:\duplicati_aux\duplicati_client.exe run 34
rem timeout /t 5 /nobreak > nul
D:\duplicati_aux\duplicati_client.exe logout
:end
exit 0

It not work with or without the time commands.

drwtsn32 · August 2, 2019, 5:03pm

You don’t literally have that word “Blockquote” at the top of your script, do you?

xblitz · August 2, 2019, 5:09pm

mmh sorry -.-’ the heat comes to my head. Of course the blockquete is a mistake because I’m not practiced with the editor of this site. If I launch the script from cmd.exe it work’s fine without error code.

Now I re-edit the post.

drwtsn32 · August 2, 2019, 5:22pm

Probably because the DUPLICATI__ environment variables are not present when you manually run it from cmd.exe – they only exist when the script is launched by Duplicati. So when you test it from cmd.exe, it ends up doing that goto end and exits right away.

Try temporarily removing those two if lines at the top and then run it from cmd.exe.

xblitz · August 2, 2019, 5:34pm

No because if I delete the first and second lines or declare the environment variables and next launch the script all works correctly. It couldn’t be a problem of “database lock” because I try to launch the script (from command line) when a backup task is still in progress and even in this case all work correctly: first the backup finish and next start the backup of “metadata”.

drwtsn32 · August 2, 2019, 5:38pm

Ok gotcha! My next suggestion is to capture the output of these commands in your script:

D:\duplicati_aux\duplicati_client.exe login   >  D:\duplicati_aux\LogFile.txt 2>&1
D:\duplicati_aux\duplicati_client.exe run 34  >> D:\duplicati_aux\LogFile.txt 2>&1
D:\duplicati_aux\duplicati_client.exe logout  >> D:\duplicati_aux\LogFile.txt 2>&1

Save the script and then let Duplicati launch it at the end of a backup job, then check the log file.

EDIT: I had to make a correction to the top line. I was using Linux-style redirection

xblitz · August 2, 2019, 5:53pm

I write a simple C program who create a text file and if I launch a batch file with this command; The result is no error and the backup jobs works. Next step: try your code! and log the activity.

drwtsn32 · August 2, 2019, 6:00pm

Also, sometimes it’s handy to put this at the top of your script so you can validate all the DUPLICATI environment variables:

set DUPLICATI > C:\Folder\DupVars.txt

Helps with debugging. But be careful there may be some sensitive info in those variables.

xblitz · August 2, 2019, 7:22pm

Ok this is the result of

set DUPLICATI

DUPLICATI__allow_sleep=false
DUPLICATI__backup_name=backup di prova
DUPLICATI__blocksize=102400b
DUPLICATI__block_hash_algorithm=SHA256
DUPLICATI__compression_module=zip
DUPLICATI__dblock_size=50mb
DUPLICATI__dbpath= CORRECT_PATH
DUPLICATI__disable_module=console-password-input
DUPLICATI__encryption_module=
DUPLICATI__EVENTNAME=AFTER
DUPLICATI__file_hash_algorithm=SHA256
DUPLICATI__LOCALPATH=D:\duplicati_aux
DUPLICATI__main_action=Backup
DUPLICATI__no_encryption=true
DUPLICATI__OPERATIONNAME=Backup
DUPLICATI__PARSED_RESULT=Success
DUPLICATI__REMOTEURL=file://D:\backup_di_prova
DUPLICATI__RESULTFILE=C:\WINDOWS\TEMP\dup-a78e93d6-46aa-46cd-bcf6-bc35eaf9bfad
DUPLICATI__run_script_after=D:\duplicati_aux\backup_database.bat
DUPLICATI__send_mail_any_operation=true
DUPLICATI__send_mail_from=sensible data
DUPLICATI__send_mail_password=sensible data
DUPLICATI__send_mail_subject=Duplicati %PARSEDRESULT%, %OPERATIONNAME% report for %backup-name%
DUPLICATI__send_mail_to=sensible data
DUPLICATI__send_mail_url=sensible data
DUPLICATI__send_mail_username=sensible data
DUPLICATI__throttle_download=
DUPLICATI__throttle_upload=
DUPLICATI__use_block_cache=true

for the log file instead I obtain a strange result: The file is correctly created but the dimension is 0KB and is empty. The only way to read the file is stop the service… but it shouldn’t be correct anyway the log is:

The request timed out. Is the server running?
Not logged in
Logging out…

of course the server is running . In addition I notice that since I had installed the software in the log there are many errors related to assembly that can’t be loaded (it’s related to microsoft windowsazure) It could be a broken installation of the .NET Framework?

drwtsn32 · August 2, 2019, 7:30pm

Oh wait… You are running Duplicati as a service? There is currently a bug where duplicati-client doesn’t work properly in that type of setup:

github.com/Pectojin/duplicati-client

Can't get duc to run from a Duplicati job's batch script

opened 01:02PM - 01 Aug 19 UTC

Taomyn

I'm trying to get duc to perform a specific backup at the end of my main backups… (Windows) - basically it's a job to separately back up the Duplicati databases whenever a backup completes. Currently I have simply exported the backup job and added it to the batch script that gets run at the end of the main jobs. This works well but has 2 draw backs, it's insecure as the credentials are all there and it doesn't update the job last run info in the GUI. Hence trying to use duc. When it runs the batch script the process gets stuck, in fact there seems to be two processes spawned with the same command line parameters. Duplicati runs as a service using the local system account and if I use a tool like "psexec" to spawn a local systsem account cmd prompt, I can run my commands so I know the syntax is good. I'm not sure if this is related but I did notice that like Duplicati, duc stores the config in the profile folder but for the local system account this is in a bad location under "system32" - it's better practice to use the "ProgramData" folder. I did however see that even with my test the folder in system32 never got created and I have no idea why.

xblitz · August 2, 2019, 7:58pm

Yes I run Duplicati as Service (I’m apologyze, I took it for granted, but I must specify this). I search in the forum in the last days but I don’t found never related this… but this bug is reported yesterday… ok, this isn’t a problem for me. I wait a new release of duplicati client (or duplicati ?) and retry my experiments.

ts678 · August 2, 2019, 7:58pm

Do we know that duplicati-client.exe can ever do backups in a --run-script-after (service or not-service)?

There are things that happen after the backup itself, such as sending job report and presumably scripts.

Duplicati seemingly won’t schedule next job until these are finished, e.g. if script waits, so does next job.
Test used a 300 second timeout. Status bar said backup was complete, yet GUI run of next job waited…

Duplicati client is (I think) just a lower-level interface into scheduling so post-backup DB backup may be waiting for itself to complete before it can launch DB backup job. Such deadlock won’t occur externally, explaining why script can run externally. In that case, backup has no run-after work, so can back up DB.

If this is the problem, a solution might be to export GUI job to backup DB, then just run that in the .bat file. Such command line work runs independently from jobs that run in the server, so you get no GUI results.

xblitz · August 2, 2019, 8:04pm

@ts678 I know this “workaround” but in this scenario we lost the control through web UI and, in my opinion, this is a strenght of duplicati.

ts678 · August 2, 2019, 8:07pm

Maybe you can use start or something to launch a background job to run duc, while post-run-script ends.

xblitz · August 2, 2019, 8:13pm

@ts678 Just to clarify: you suggest to create a script containing only the start command and launch this script with --run-script-after. The start command run my real backup database script. Correct?

EDIT: This is a joke… but it works!

drwtsn32 · August 2, 2019, 9:11pm

Yep, I do it and it works great!

drwtsn32 · August 2, 2019, 9:12pm

Wait really, it did? Strange that it would make a difference!

ts678 · August 2, 2019, 9:13pm

Beats me. I don’t even have duc, but I posted the info to the GitHub issue and Pectojin can speculate on it.