Hi All,
I wanted to give you a watchdog script for Win11 I’m using for Duplicati.
The description is in the code header.
Have fun!
EDIT: Code change for PowerShell 7 only
#requires -Version 7.0
<#
.SYNOPSIS
Duplicati 2.x watchdog for Windows using PowerShell 7+.
.DESCRIPTION
This script is designed for use as Duplicati's "run-script-before"
command.
The run-script-before process does not perform the monitoring itself.
It starts an independent hidden PowerShell 7 child process and exits
immediately with exit code 0.
The independent watchdog child:
1. Reads DUPLICATI__backup_name.
2. Resolves the exact Duplicati BackupID.
3. Waits for the corresponding backup task to become active.
4. Obtains the exact TaskID from:
ServerUtil status --json
5. Verifies that exact TaskID through:
GET /api/v1/tasks
6. Monitors only that exact TaskID.
7. Exits immediately when that exact task finishes.
8. If the task exceeds TimeoutMinutes:
- sends STOP to the exact TaskID
- waits StopWaitSeconds
- verifies the exact same BackupID/TaskID
- sends ABORT if the same task is still active
- waits AbortWaitSeconds
- verifies the exact same BackupID/TaskID again
- terminates Duplicati.Server.exe only if the same task
is still active
- waits ServerRestartWaitSeconds
- starts Duplicati.Server.exe again if necessary
SAFETY PRINCIPLE
No STOP, ABORT or server termination is performed merely because
some Duplicati task exists.
Immediately before every destructive operation the watchdog verifies:
BackupID == originally resolved BackupID
TaskID == originally resolved TaskID
The REST API must additionally report that exact TaskID as Running.
If the BackupID or TaskID changes, or the REST API no longer reports
the exact task as Running, destructive action is cancelled.
MULTIPLE WATCHDOGS
Every watchdog is associated with one exact BackupID and TaskID.
This allows several backup jobs and watchdog processes to coexist.
NORMAL COMPLETION
The watchdog exits as soon as its exact task is no longer Running.
It does not wait for the complete timeout period.
BACKUP IDENTIFICATION
The backup name is supplied by Duplicati through:
DUPLICATI__backup_name
The exact name is resolved through:
GET /api/v1/backups
TASK IDENTIFICATION
ServerUtil status --json is the authoritative source for:
BackupID -> TaskID
The REST API /api/v1/tasks is then used to verify the exact TaskID
and its current state.
AUTHENTICATION
The Duplicati server password is read from Windows Credential Manager
using the target:
duplicati-serverutil-password
ServerUtil is called using:
--password=$duplicati-serverutil-password
--secret-provider=wincred://
The password is never stored in this script.
POWERSHELL VERSION
PowerShell 7.x is required.
Windows PowerShell 5.1 may remain installed on the system. It is not
used by this script.
The run-script-before command must explicitly invoke PowerShell 7.
Example:
"C:\Program Files\PowerShell\7\pwsh.exe" -NoProfile -ExecutionPolicy Bypass -File "C:\Path\To\DuplicatiWatchdog.ps1"
The exact script path is installation-specific and must be adjusted
outside this script.
CHILD PROCESS
The parent process determines the PowerShell 7 executable from
$PSHOME and starts an independent hidden child.
The child receives:
-WatchdogChild
The child inherits Duplicati's environment, including:
DUPLICATI__backup_name
The parent does not wait for the child.
TEST MODE
Test mode is non-destructive.
.\DuplicatiWatchdog.ps1 -Test -Debug -BackupName "backup-name"
Test mode verifies:
- Windows Credential Manager access
- Duplicati API authentication
- exact backup-name lookup
- BackupID resolution
- ServerUtil active-task lookup
- exact TaskID resolution
- REST TaskID verification
Test mode does NOT:
- STOP a task
- ABORT a task
- terminate Duplicati.Server.exe
- restart Duplicati.Server.exe
IMPORTANT
A successful test proves only the checks performed by the test.
It does not prove that STOP, ABORT or server restart will succeed.
DEBUG MODE
Use:
-Debug
to display diagnostic information.
No permanent log file is created.
DEFAULT CONFIGURATION
TimeoutMinutes = 10
PollIntervalSeconds = 5
StopWaitSeconds = 10
AbortWaitSeconds = 10
ServerRestartWaitSeconds = 15
InitialDiscoverySeconds = 60
SERVER RESTART
The watchdog only starts Duplicati.Server.exe after termination if
the server process has not already returned.
SECURITY
The Duplicati password is stored outside this script in Windows
Credential Manager.
The script does not write the password to a log file.
INSTALLATION
Save this script as:
DuplicatiWatchdog.ps1
Place it in any directory accessible to the account running Duplicati.
Configure Duplicati's run-script-before command to explicitly use
PowerShell 7.
MANUAL TEST
Example:
.\DuplicatiWatchdog.ps1 -Test -Debug -BackupName "backup-name"
LICENSE / PUBLICATION
This script is provided as automation code for Duplicati installations.
Use and modify it at your own risk. No warranty is provided.
#>
[CmdletBinding()]
param(
[Alias('Test')]
[switch]$TestMode,
[switch]$WatchdogChild,
[string]$BackupName
)
Set-StrictMode -Version Latest
$ErrorActionPreference = 'Stop'
# ============================================================================
# CONFIGURATION
# ============================================================================
$DuplicatiServerUrl = 'http://127.0.0.1:8200'
$ServerUtilPath = Join-Path `
${env:ProgramFiles} `
'Duplicati 2\Duplicati.CommandLine.ServerUtil.exe'
$DuplicatiServerExe = Join-Path `
${env:ProgramFiles} `
'Duplicati 2\Duplicati.Server.exe'
$CredentialTarget = 'duplicati-serverutil-password'
$TimeoutMinutes = 10
$PollIntervalSeconds = 5
$StopWaitSeconds = 10
$AbortWaitSeconds = 10
$ServerRestartWaitSeconds = 15
$InitialDiscoverySeconds = 60
# ============================================================================
# DEBUG OUTPUT
# ============================================================================
function Write-DebugMessage {
param(
[Parameter(Mandatory)]
[string]$Message
)
if ($DebugPreference -ne 'SilentlyContinue') {
Write-Host "[DEBUG $(Get-Date -Format 'yyyy-MM-dd HH:mm:ss.fff')] $Message"
}
}
# ============================================================================
# WINDOWS CREDENTIAL MANAGER
# ============================================================================
Add-Type @'
using System;
using System.Runtime.InteropServices;
public static class NativeCredentialManager
{
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
public struct CREDENTIAL
{
public uint Flags;
public uint Type;
public IntPtr TargetName;
public IntPtr Comment;
public System.Runtime.InteropServices.ComTypes.FILETIME LastWritten;
public uint CredentialBlobSize;
public IntPtr CredentialBlob;
public uint Persist;
public uint AttributeCount;
public IntPtr Attributes;
public IntPtr TargetAlias;
public IntPtr UserName;
}
[DllImport("advapi32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
public static extern bool CredRead(
string target,
uint type,
uint flags,
out IntPtr credential
);
[DllImport("advapi32.dll", SetLastError = true)]
public static extern void CredFree(IntPtr credential);
}
'@
function Get-WindowsCredentialPassword {
param(
[Parameter(Mandatory)]
[string]$Target
)
Write-DebugMessage "Reading Windows Credential Manager target '$Target'."
$credentialPtr = [IntPtr]::Zero
try {
$success = [NativeCredentialManager]::CredRead(
$Target,
1,
0,
[ref]$credentialPtr
)
if (-not $success) {
$errorCode = [Runtime.InteropServices.Marshal]::GetLastWin32Error()
throw "CredRead failed for '$Target'. Windows error code: $errorCode."
}
$credential = [Runtime.InteropServices.Marshal]::PtrToStructure(
$credentialPtr,
[type][NativeCredentialManager+CREDENTIAL]
)
if ($credential.CredentialBlobSize -eq 0) {
throw "Credential '$Target' exists but contains an empty password."
}
$password = [Runtime.InteropServices.Marshal]::PtrToStringUni(
$credential.CredentialBlob,
[int]($credential.CredentialBlobSize / 2)
)
if ([string]::IsNullOrWhiteSpace($password)) {
throw "Credential '$Target' contains an empty password."
}
Write-DebugMessage 'Credential Manager lookup succeeded.'
return $password
}
finally {
if ($credentialPtr -ne [IntPtr]::Zero) {
[NativeCredentialManager]::CredFree($credentialPtr)
}
}
}
# ============================================================================
# DUPLICATI REST API
# ============================================================================
function Invoke-DuplicatiApi {
param(
[Parameter(Mandatory)]
[ValidateSet('GET', 'POST', 'PUT', 'DELETE')]
[string]$Method,
[Parameter(Mandatory)]
[string]$Path,
[string]$AccessToken,
[object]$Body
)
$uri = "$DuplicatiServerUrl$Path"
$headers = @{
Accept = 'application/json'
}
if (-not [string]::IsNullOrWhiteSpace($AccessToken)) {
$headers['Authorization'] = "Bearer $AccessToken"
}
Write-DebugMessage "API $Method $uri"
$params = @{
Uri = $uri
Method = $Method
Headers = $headers
ErrorAction = 'Stop'
}
if ($null -ne $Body) {
$params['ContentType'] = 'application/json'
$params['Body'] = ($Body | ConvertTo-Json -Compress)
}
try {
return Invoke-RestMethod @params
}
catch {
Write-DebugMessage "API request failed: $($_.Exception.Message)"
throw
}
}
# ============================================================================
# DUPLICATI AUTHENTICATION
# ============================================================================
function Get-DuplicatiAccessToken {
param(
[Parameter(Mandatory)]
[string]$Password
)
Write-DebugMessage 'Requesting Duplicati REST access token.'
$body = @{
password = $Password
}
$response = Invoke-DuplicatiApi `
-Method POST `
-Path '/api/v1/auth/login' `
-Body $body
$token = $null
foreach ($propertyName in @(
'AccessToken',
'accessToken',
'Token',
'token'
)) {
if ($null -ne $response.PSObject.Properties[$propertyName]) {
$candidate = $response.$propertyName
if (-not [string]::IsNullOrWhiteSpace([string]$candidate)) {
$token = [string]$candidate
break
}
}
}
if ([string]::IsNullOrWhiteSpace($token)) {
throw 'Duplicati authentication succeeded but no access token was returned.'
}
Write-DebugMessage 'Duplicati REST access token obtained.'
return $token
}
# ============================================================================
# EXACT BACKUP LOOKUP
# ============================================================================
function Get-ExactBackup {
param(
[Parameter(Mandatory)]
[string]$AccessToken,
[Parameter(Mandatory)]
[string]$Name
)
Write-DebugMessage "Looking up exact backup name '$Name'."
$response = Invoke-DuplicatiApi `
-Method GET `
-Path '/api/v1/backups' `
-AccessToken $AccessToken
$items = @($response)
Write-DebugMessage "Backup response top-level item count: $($items.Count)"
$matches = @()
foreach ($item in $items) {
if ($null -eq $item) {
continue
}
$id = $null
$candidateName = $null
if ($null -ne $item.PSObject.Properties['ID']) {
$id = [string]$item.ID
}
elseif ($null -ne $item.PSObject.Properties['Id']) {
$id = [string]$item.Id
}
if ($null -ne $item.PSObject.Properties['Name']) {
$candidateName = [string]$item.Name
}
Write-DebugMessage "Backup candidate: ID='$id' Name='$candidateName'"
if ($candidateName -ceq $Name) {
$matches += $item
}
}
if ($matches.Count -eq 0) {
throw "No backup with the exact name '$Name' was found."
}
if ($matches.Count -gt 1) {
throw "More than one backup with the exact name '$Name' was returned."
}
$backup = $matches[0]
$backupId = $null
if ($null -ne $backup.PSObject.Properties['ID']) {
$backupId = [string]$backup.ID
}
elseif ($null -ne $backup.PSObject.Properties['Id']) {
$backupId = [string]$backup.Id
}
if ([string]::IsNullOrWhiteSpace($backupId)) {
throw "Backup '$Name' was found but has no usable BackupID."
}
return [pscustomobject]@{
Name = $Name
ID = $backupId
}
}
# ============================================================================
# SERVERUTIL STATUS
# ============================================================================
function Get-ServerUtilStatus {
if (-not (Test-Path -LiteralPath $ServerUtilPath)) {
throw "ServerUtil was not found: $ServerUtilPath"
}
Write-DebugMessage 'Calling ServerUtil status --json.'
$arguments = @(
'status'
'--json'
'--password=$duplicati-serverutil-password'
'--secret-provider=wincred://'
)
$output = & $ServerUtilPath @arguments 2>&1
$exitCode = $LASTEXITCODE
Write-DebugMessage "ServerUtil exit code: $exitCode"
$raw = ($output | Out-String).Trim()
Write-DebugMessage "ServerUtil raw response:`n$raw"
if ($exitCode -ne 0) {
throw "ServerUtil status failed with exit code $exitCode."
}
if ([string]::IsNullOrWhiteSpace($raw)) {
throw 'ServerUtil returned an empty response.'
}
try {
return ($raw | ConvertFrom-Json)
}
catch {
throw "Could not parse ServerUtil JSON: $($_.Exception.Message)"
}
}
# ============================================================================
# EXACT ACTIVE TASK
# ============================================================================
function Find-ExactActiveTask {
param(
[Parameter(Mandatory)]
[string]$BackupId
)
$status = Get-ServerUtilStatus
if ($null -eq $status.ActiveTask) {
Write-DebugMessage 'ServerUtil reports no active task.'
return $null
}
$activeTaskId = [string]$status.ActiveTask.Task
$activeBackupId = [string]$status.ActiveTask.BackupId
Write-DebugMessage "ServerUtil ActiveTask: Task='$activeTaskId' BackupId='$activeBackupId'."
if ($activeBackupId -ne $BackupId) {
Write-DebugMessage "Active task belongs to BackupID '$activeBackupId', not '$BackupId'."
return $null
}
if ([string]::IsNullOrWhiteSpace($activeTaskId)) {
throw "ServerUtil returned BackupID '$BackupId' but no TaskID."
}
return [pscustomobject]@{
BackupId = $BackupId
TaskId = $activeTaskId
}
}
# ============================================================================
# EXACT REST TASK
# ============================================================================
function Get-ExactTaskState {
param(
[Parameter(Mandatory)]
[string]$AccessToken,
[Parameter(Mandatory)]
[string]$TaskId
)
$response = Invoke-DuplicatiApi `
-Method GET `
-Path '/api/v1/tasks' `
-AccessToken $AccessToken
$records = @($response)
foreach ($record in $records) {
if ($null -eq $record) {
continue
}
if ($null -eq $record.PSObject.Properties['ID']) {
continue
}
if ([string]$record.ID -eq $TaskId) {
return $record
}
}
return $null
}
function Test-ExactTaskActive {
param(
[Parameter(Mandatory)]
[string]$AccessToken,
[Parameter(Mandatory)]
[string]$TaskId
)
$task = Get-ExactTaskState `
-AccessToken $AccessToken `
-TaskId $TaskId
if ($null -eq $task) {
Write-DebugMessage "REST API has no record for TaskID '$TaskId'."
return $null
}
Write-DebugMessage "REST task state:`n$($task | ConvertTo-Json -Depth 10)"
if ([string]$task.Status -eq 'Running') {
return $task
}
return $null
}
# ============================================================================
# FINAL SAFETY CHECK
# ============================================================================
function Confirm-ExactTaskStillActive {
param(
[Parameter(Mandatory)]
[string]$BackupId,
[Parameter(Mandatory)]
[string]$TaskId
)
Write-DebugMessage "Safety verification for BackupID '$BackupId', TaskID '$TaskId'."
$status = Get-ServerUtilStatus
if ($null -eq $status.ActiveTask) {
Write-DebugMessage 'Safety verification failed: no active task.'
return $false
}
$currentTaskId = [string]$status.ActiveTask.Task
$currentBackupId = [string]$status.ActiveTask.BackupId
Write-DebugMessage "Safety check: BackupID='$currentBackupId', TaskID='$currentTaskId'."
if ($currentBackupId -ne $BackupId) {
Write-DebugMessage 'Safety verification failed: BackupID changed.'
return $false
}
if ($currentTaskId -ne $TaskId) {
Write-DebugMessage 'Safety verification failed: TaskID changed.'
return $false
}
$taskState = Test-ExactTaskActive `
-AccessToken $script:AccessToken `
-TaskId $TaskId
if ($null -eq $taskState) {
Write-DebugMessage 'Safety verification failed: REST task is no longer Running.'
return $false
}
return $true
}
# ============================================================================
# STOP
# ============================================================================
function Stop-ExactTask {
param(
[Parameter(Mandatory)]
[string]$AccessToken,
[Parameter(Mandatory)]
[string]$TaskId
)
Write-DebugMessage "Sending STOP request for TaskID $TaskId."
try {
Invoke-DuplicatiApi `
-Method POST `
-Path "/api/v1/task/$TaskId/stop" `
-AccessToken $AccessToken | Out-Null
Write-DebugMessage "STOP request accepted for TaskID $TaskId."
}
catch {
Write-DebugMessage "STOP request failed for TaskID ${TaskId}: $($_.Exception.Message)"
throw
}
}
# ============================================================================
# ABORT
# ============================================================================
function Abort-ExactTask {
param(
[Parameter(Mandatory)]
[string]$AccessToken,
[Parameter(Mandatory)]
[string]$TaskId
)
Write-DebugMessage "Sending ABORT request for TaskID $TaskId."
try {
Invoke-DuplicatiApi `
-Method POST `
-Path "/api/v1/task/$TaskId/abort" `
-AccessToken $AccessToken | Out-Null
Write-DebugMessage "ABORT request accepted for TaskID $TaskId."
}
catch {
Write-DebugMessage "ABORT request failed for TaskID ${TaskId}: $($_.Exception.Message)"
throw
}
}
# ============================================================================
# DUPLICATI SERVER PROCESS
# ============================================================================
function Get-DuplicatiServerProcess {
$processes = @(
Get-Process `
-Name 'Duplicati.Server' `
-ErrorAction SilentlyContinue
)
if ($processes.Count -eq 0) {
return $null
}
return $processes[0]
}
function Stop-DuplicatiServer {
$process = Get-DuplicatiServerProcess
if ($null -eq $process) {
Write-DebugMessage 'Duplicati.Server.exe is not currently running.'
return
}
Write-DebugMessage "Terminating Duplicati.Server.exe PID $($process.Id)."
Stop-Process `
-Id $process.Id `
-Force `
-ErrorAction Stop
Write-DebugMessage 'Duplicati.Server.exe terminated.'
}
# ============================================================================
# SERVER START
# ============================================================================
function Start-DuplicatiServer {
if (-not (Test-Path -LiteralPath $DuplicatiServerExe)) {
throw "Duplicati.Server.exe was not found: $DuplicatiServerExe"
}
$existing = Get-DuplicatiServerProcess
if ($null -ne $existing) {
Write-DebugMessage "Duplicati.Server.exe already running with PID $($existing.Id)."
return
}
Write-DebugMessage 'Starting Duplicati.Server.exe.'
$psi = [System.Diagnostics.ProcessStartInfo]::new()
$psi.FileName = $DuplicatiServerExe
$psi.UseShellExecute = $false
$psi.CreateNoWindow = $true
$psi.WorkingDirectory = Split-Path -Parent $DuplicatiServerExe
$process = [System.Diagnostics.Process]::new()
$process.StartInfo = $psi
if (-not $process.Start()) {
throw 'Failed to start Duplicati.Server.exe.'
}
Write-DebugMessage "Duplicati.Server.exe started with PID $($process.Id)."
$process.Dispose()
}
function Wait-ForDuplicatiServer {
param(
[Parameter(Mandatory)]
[int]$TimeoutSeconds
)
$deadline = (Get-Date).AddSeconds($TimeoutSeconds)
while ((Get-Date) -lt $deadline) {
try {
$status = Get-ServerUtilStatus
if ([string]$status.ServerState -eq 'Running') {
Write-DebugMessage 'Duplicati server reports Running.'
return $true
}
}
catch {
Write-DebugMessage "Server not ready yet: $($_.Exception.Message)"
}
Start-Sleep -Seconds 1
}
return $false
}
# ============================================================================
# START INDEPENDENT WATCHDOG CHILD
# ============================================================================
function Start-WatchdogChild {
Write-DebugMessage 'Starting independent PowerShell 7 watchdog child.'
$pwshPath = Join-Path $PSHOME 'pwsh.exe'
if (-not (Test-Path -LiteralPath $pwshPath)) {
throw "PowerShell 7 executable was not found: $pwshPath"
}
if ([string]::IsNullOrWhiteSpace($PSCommandPath)) {
throw 'PSCommandPath is empty; cannot start watchdog child.'
}
$psi = [System.Diagnostics.ProcessStartInfo]::new()
$psi.FileName = $pwshPath
$psi.UseShellExecute = $false
$psi.CreateNoWindow = $true
$psi.WindowStyle = [System.Diagnostics.ProcessWindowStyle]::Hidden
$psi.WorkingDirectory = Split-Path `
-Parent `
$PSCommandPath
[void]$psi.ArgumentList.Add('-NoProfile')
[void]$psi.ArgumentList.Add('-NonInteractive')
[void]$psi.ArgumentList.Add('-ExecutionPolicy')
[void]$psi.ArgumentList.Add('Bypass')
[void]$psi.ArgumentList.Add('-File')
[void]$psi.ArgumentList.Add($PSCommandPath)
[void]$psi.ArgumentList.Add('-WatchdogChild')
$process = [System.Diagnostics.Process]::new()
$process.StartInfo = $psi
if (-not $process.Start()) {
throw 'PowerShell 7 watchdog child could not be started.'
}
Write-DebugMessage "Watchdog child started with PID $($process.Id)."
$process.Dispose()
}
# ============================================================================
# NON-DESTRUCTIVE TEST MODE
# ============================================================================
function Invoke-TestMode {
param(
[Parameter(Mandatory)]
[string]$Name
)
Write-Host ''
Write-Host '================================================'
Write-Host 'Duplicati Watchdog - NON-DESTRUCTIVE TEST'
Write-Host '================================================'
Write-Host ''
Write-Host '[1/5] Reading password from Windows Credential Manager...'
$password = Get-WindowsCredentialPassword `
-Target $CredentialTarget
Write-Host ' Credential Manager OK'
Write-Host ''
Write-Host '[2/5] Authenticating against Duplicati API...'
$script:AccessToken = Get-DuplicatiAccessToken `
-Password $password
Write-Host ' API authentication OK'
Write-Host ''
Write-Host '[3/5] Resolving exact backup name...'
$backup = Get-ExactBackup `
-AccessToken $script:AccessToken `
-Name $Name
Write-Host ''
Write-Host ' EXACT BACKUP FOUND'
Write-Host ''
Write-Host " BackupName = $($backup.Name)"
Write-Host " BackupID = $($backup.ID)"
Write-Host ''
Write-Host '[4/5] Resolving exact active task through ServerUtil...'
$activeTask = Find-ExactActiveTask `
-BackupId $backup.ID
if ($null -eq $activeTask) {
Write-Host ' No active task found for this backup.'
Write-Host ''
Write-Host 'TEST PASSED - API and backup resolution work; no active task existed to verify.'
return
}
Write-Host ''
Write-Host ' EXACT ACTIVE TASK FOUND'
Write-Host ''
Write-Host " BackupName = $($backup.Name)"
Write-Host " BackupID = $($activeTask.BackupId)"
Write-Host " TaskID = $($activeTask.TaskId)"
Write-Host ''
Write-Host '[5/5] Verifying exact TaskID through REST API...'
$taskState = Get-ExactTaskState `
-AccessToken $script:AccessToken `
-TaskId $activeTask.TaskId
if ($null -eq $taskState) {
throw "TaskID $($activeTask.TaskId) was not returned by /api/v1/tasks."
}
Write-Host ''
Write-Host ' REST task state:'
Write-Host ($taskState | ConvertTo-Json -Depth 10)
if ([string]$taskState.ID -ne [string]$activeTask.TaskId) {
throw 'REST API TaskID does not match ServerUtil TaskID.'
}
if ([string]$taskState.Status -ne 'Running') {
throw "Exact TaskID $($activeTask.TaskId) is not Running."
}
Write-Host ''
Write-Host 'TEST PASSED - exact BackupID -> TaskID resolution and REST verification work.'
Write-Host ''
Write-Host 'IMPORTANT: This test is NON-DESTRUCTIVE.'
Write-Host 'STOP, ABORT and server restart were NOT executed.'
}
# ============================================================================
# WATCHDOG
# ============================================================================
function Invoke-Watchdog {
param(
[Parameter(Mandatory)]
[string]$Name
)
Write-DebugMessage "Watchdog child started for backup '$Name'."
$password = Get-WindowsCredentialPassword `
-Target $CredentialTarget
$script:AccessToken = Get-DuplicatiAccessToken `
-Password $password
$backup = Get-ExactBackup `
-AccessToken $script:AccessToken `
-Name $Name
$backupId = [string]$backup.ID
Write-DebugMessage "Resolved backup '$Name' to BackupID '$backupId'."
# ------------------------------------------------------------------------
# Wait for this backup to become the active task.
# ------------------------------------------------------------------------
$discoveryDeadline = (Get-Date).AddSeconds($InitialDiscoverySeconds)
$exactTask = $null
while ((Get-Date) -lt $discoveryDeadline) {
$exactTask = Find-ExactActiveTask `
-BackupId $backupId
if ($null -ne $exactTask) {
break
}
Start-Sleep -Seconds $PollIntervalSeconds
}
if ($null -eq $exactTask) {
Write-DebugMessage `
"No active task for BackupID '$backupId' was discovered within $InitialDiscoverySeconds seconds."
return
}
$taskId = [string]$exactTask.TaskId
Write-DebugMessage `
"Monitoring exact BackupID '$backupId', TaskID '$taskId'."
# ------------------------------------------------------------------------
# Initial REST verification.
# ------------------------------------------------------------------------
$taskState = Test-ExactTaskActive `
-AccessToken $script:AccessToken `
-TaskId $taskId
if ($null -eq $taskState) {
Write-DebugMessage `
"TaskID '$taskId' was no longer Running during initial verification."
return
}
# ------------------------------------------------------------------------
# Determine task start time.
# ------------------------------------------------------------------------
$taskStartTime = $null
if ($null -ne $taskState.TaskStarted) {
try {
$taskStartTime = [DateTimeOffset]::Parse(
[string]$taskState.TaskStarted
)
}
catch {
Write-DebugMessage `
'Could not parse TaskStarted; using current time.'
}
}
if ($null -eq $taskStartTime) {
$taskStartTime = [DateTimeOffset]::Now
}
Write-DebugMessage "Task start time: $taskStartTime"
$timeoutAt = $taskStartTime.AddMinutes($TimeoutMinutes)
Write-DebugMessage "Timeout threshold: $timeoutAt"
# ------------------------------------------------------------------------
# Monitor exact TaskID.
# ------------------------------------------------------------------------
while ($true) {
$currentTask = Test-ExactTaskActive `
-AccessToken $script:AccessToken `
-TaskId $taskId
if ($null -eq $currentTask) {
Write-DebugMessage `
"Exact TaskID '$taskId' is no longer Running. Watchdog exits normally."
return
}
if ((Get-Date) -lt $timeoutAt.LocalDateTime) {
Start-Sleep `
-Seconds $PollIntervalSeconds
continue
}
Write-DebugMessage `
"TIMEOUT reached for exact TaskID '$taskId'."
break
}
# ------------------------------------------------------------------------
# STOP
# ------------------------------------------------------------------------
Write-DebugMessage 'Performing final safety check before STOP.'
if (-not (Confirm-ExactTaskStillActive `
-BackupId $backupId `
-TaskId $taskId)) {
Write-DebugMessage `
'STOP cancelled because the exact task is no longer safely identifiable.'
return
}
Stop-ExactTask `
-AccessToken $script:AccessToken `
-TaskId $taskId
Start-Sleep `
-Seconds $StopWaitSeconds
# ------------------------------------------------------------------------
# Verify after STOP.
# ------------------------------------------------------------------------
Write-DebugMessage 'Checking exact task after STOP.'
$stillExact = Confirm-ExactTaskStillActive `
-BackupId $backupId `
-TaskId $taskId
if (-not $stillExact) {
Write-DebugMessage `
"TaskID '$taskId' stopped successfully. Watchdog exits."
return
}
# ------------------------------------------------------------------------
# ABORT
# ------------------------------------------------------------------------
Write-DebugMessage `
"TaskID '$taskId' is still active after STOP."
if (-not (Confirm-ExactTaskStillActive `
-BackupId $backupId `
-TaskId $taskId)) {
Write-DebugMessage `
'ABORT cancelled because the exact task is no longer safely identifiable.'
return
}
Abort-ExactTask `
-AccessToken $script:AccessToken `
-TaskId $taskId
Start-Sleep `
-Seconds $AbortWaitSeconds
# ------------------------------------------------------------------------
# Verify after ABORT.
# ------------------------------------------------------------------------
Write-DebugMessage 'Checking exact task after ABORT.'
$stillExact = Confirm-ExactTaskStillActive `
-BackupId $backupId `
-TaskId $taskId
if (-not $stillExact) {
Write-DebugMessage `
"TaskID '$taskId' terminated after ABORT. Watchdog exits."
return
}
# ------------------------------------------------------------------------
# LAST RESORT: TERMINATE SERVER
# ------------------------------------------------------------------------
Write-DebugMessage `
"TaskID '$taskId' is still active after ABORT."
if (-not (Confirm-ExactTaskStillActive `
-BackupId $backupId `
-TaskId $taskId)) {
Write-DebugMessage `
'Server termination cancelled because the exact task is no longer safely identifiable.'
return
}
Write-DebugMessage `
"Exact TaskID '$taskId' is still active. Terminating Duplicati.Server.exe."
Stop-DuplicatiServer
Write-DebugMessage `
"Waiting $ServerRestartWaitSeconds seconds before checking/restarting Duplicati."
Start-Sleep `
-Seconds $ServerRestartWaitSeconds
# ------------------------------------------------------------------------
# Restart only if the server did not return by itself.
# ------------------------------------------------------------------------
$serverProcess = Get-DuplicatiServerProcess
if ($null -eq $serverProcess) {
Write-DebugMessage `
'Duplicati.Server.exe has not returned. Starting it.'
Start-DuplicatiServer
if (Wait-ForDuplicatiServer -TimeoutSeconds 30) {
Write-DebugMessage `
'Duplicati server restart confirmed.'
}
else {
Write-DebugMessage `
'WARNING: Duplicati server did not report Running within 30 seconds.'
}
}
else {
Write-DebugMessage `
"Duplicati.Server.exe already returned with PID $($serverProcess.Id). No second start performed."
}
Write-DebugMessage 'Watchdog escalation completed.'
}
# ============================================================================
# ENTRY POINT
# ============================================================================
try {
# ------------------------------------------------------------------------
# TEST MODE
#
# Test mode must work without DUPLICATI__backup_name.
# ------------------------------------------------------------------------
if ($TestMode) {
if ([string]::IsNullOrWhiteSpace($BackupName)) {
throw 'Test mode requires -BackupName "exact backup name".'
}
Invoke-TestMode `
-Name $BackupName
exit 0
}
# ------------------------------------------------------------------------
# WATCHDOG CHILD
# ------------------------------------------------------------------------
if ($WatchdogChild) {
$environmentBackupName = $env:DUPLICATI__backup_name
if ([string]::IsNullOrWhiteSpace($environmentBackupName)) {
Write-DebugMessage `
'Watchdog child has no DUPLICATI__backup_name. Exiting safely.'
exit 0
}
Invoke-Watchdog `
-Name $environmentBackupName
exit 0
}
# ------------------------------------------------------------------------
# NORMAL run-script-before PARENT
#
# The parent starts the independent child and exits immediately.
# It does not monitor the child and does not wait for it.
# ------------------------------------------------------------------------
$environmentBackupName = $env:DUPLICATI__backup_name
if ([string]::IsNullOrWhiteSpace($environmentBackupName)) {
Write-DebugMessage `
'DUPLICATI__backup_name is not set. Nothing to monitor.'
exit 0
}
Write-DebugMessage `
"run-script-before invoked for backup '$environmentBackupName'."
Start-WatchdogChild
Write-DebugMessage `
'Independent watchdog child started. Parent exits immediately with code 0.'
exit 0
}
catch {
Write-Error $_.Exception.Message
# ------------------------------------------------------------------------
# The parent must never abort a Duplicati backup because the watchdog
# itself failed to start.
#
# Test mode and the child return non-zero so errors are visible when
# manually testing/debugging.
# ------------------------------------------------------------------------
if ($TestMode -or $WatchdogChild) {
exit 1
}
exit 0
}