Warning Folder X:/ Does Not Exist

Support
1

Hi,

I use Duplicati on Windows 10 to backup to an external hdd in my docking station and usb drive at two separate places.

When my backup device is not attached, Duplicati will naturally pop-up with the warning/error that it did not find the target device. In the logs: Duplicati.Library.Interface.FolderMissingException: The folder X:\ does not exist

Is there an option to hide this pop-up (just the pop-up not per se the log)? The pop-up distracts me during my work and I was hoping for a method to hide it. Does this option exist and I did not find it or do you have a recommendation to prevent such messages to pop-up?

If this option or another method does not exist, I would like to make a Github feature request and hope that if other people also desire this, it might get implemented in the future :slight_smile:

Thanks in advance for thinking about this idea and Duplicati is great!

2

A similar question was asked recently, and this was my suggestion:

3

Thanks for pointing that out! I have written my first windows batch file yesterday :slight_smile:

@ECHO OFF
ECHO Required Start Script

IF EXIST x:\ ( GOTO yes ) ELSE ( GOTO no )

:yes
ECHO X does exist
exit 0

:no
ECHO X does not exist
exit 1

This solves my biggest issue of the warning popping up around my task bar when Duplicati failed to find its scheduled target. I am wondering about the expected result however, between --run-script-before and --run-script-before-required.

  • Using the batch script as argument in --run-script-before-required: The Duplicati icon in the very right part of the task bar turns red when the script throws the exit code of 1. When opening the webserver I have to dismiss the error with header “Error while running Test Script” and text “The script “C:\Duplicati\check_if_X_exists.bat” returned with exit code 1”. I did not expect this error since I thought that the exit code would exit the backup without raising an error.
  • With --run-script-before: The taskbar icon does not turn red and there is no error in the webserver. All exactly as I like since I only want warnings/errors to be raised when other things go wrong. However, the information of the backup in the home tab registers the moment the script ran as a successful backup and the source and backup size change to 0. When connecting the drive and running the backup method it will change without problems to the real state of the backup on that drive. This is not nice since I cannot see when my last successful backup has ran.

Is the above intended behaviour? I expected the exit code 1 to abort the backup without raising any error. The difference between --run-script-before and --run-script-before-required looks to be already incorporated in the different exit codes 0-5, but this does not seem to be true.

4

That’s how I understand it is supposed to work.

Yes, from what I understand the --run-script-before-required isn’t needed any more with enhancements made to --run-script-before a while back.

Which version of Duplicati are you using?

5

I use Duplicati - 2.0.5.1_beta_2020-01-18.

From what I remember it did not matter when I used:

exit /B 1

as exit code. This could sort of affect whether windows continues in its current CMD window, thus might affect the sequencing in the duplicati software. Note that this was just a hunch that I tried since I was not fully sure of the script ending and various batch information sources when I was creating the script.

6

I don’t think you need the /b - from what I recall that only affects nested batch files.

I’ll try to do some testing on this error level thing and get back to you!

7

Great, thanks a lot drwtsn22!

When I can do anything from my side such as better documenting the process, perform some tests, etc; let me know. I also have an old laptop laying around somewhere, so I could do some tests on a complete fresh windows + duplicati install if necessary, but it will take me about a week to make the time for that.

8

I did a quick test with 2.0.5.1. Created a test backup job and ran it once to confirm it works. Then edited the config and added --run-script-before = c:\dup.cmd

In this dup.cmd script I only added a single line: exit 1

Ran the backup again and it stopped immediately with no warnings or errors. In the Live Log I can see the following:

Oct 9, 2020 11:53 AM: Aborting operation by request, requested result: Normal

So… at first glance it seems to be working for me.

Are you sure your test is working? Try just running this from cmd directly with the USB drive connected and not connected:

IF EXIST X:\ ( ECHO Exists ) ELSE ( ECHO Doesn't Exist )
9

When running IF EXIST X:\ ( ECHO Exists ) ELSE ( ECHO Doesn't Exist ), it works as expected; returns Exists when the drive is attached and returns Doesn’t Exist when the drive is not attached.

Run-script-before

I tried to recreate my problem in a minimal configuration.

Exit code 0

I expect everything to work and want to present the initial backup state before changing exit codes.

  • Create new backup in webserver, no encryption, source folder with 2 jpg files, repeat backup every 10 minutes, and use advanced setting: run-script-before C:\Duplicati\test_exit.bat. With test_exit.bat:
exit 0
  • I manually run the backup:
Last successful backup:
    Today at 12:09 PM (took 00:00:01)
    Run now 
Next scheduled run:
    Today at 12:15 PM
Source:
    374.13 KB
Backup:
    377.50 KB / 1 Version
  • I manually add a file to the source folder and wait for automatic backup to happen. Result 6+ minutes later:
Last successful backup:
    Today at 12:15 PM (took 00:00:01)
    Run now 
Next scheduled run:
    Today at 12:25 PM
Source:
    454.73 KB
Backup:
    459.50 KB / 2 Versions

Thus everything is working as expected.

Exit code 1

  • I add a file to the folder and change test_exit.bat to:
exit 1

The next automatic backup results in

Last successful backup:
    Today at 12:25 PM (took 00:00:00)
    Run now 
Next scheduled run:
    Today at 12:35 PM
Source:
    0 bytes
Backup:
    0 bytes / 0 Versions

No error or warning signals were returned.
This is, however, not exactly as I expected since the information in my dashboard is lost regarding when the last successful backup ran, that is successful in my eyes. It looks like it ran some kind of dry backup.

Looking at the Live Log, the last message looks as expected, there is a whole list of messages before, I have copied only the last part, but I can post the rest as well:

Oct 11, 2020 12:25 PM: Aborting operation by request, requested result: Normal
Oct 11, 2020 12:15 PM: The operation Backup has completed
Oct 11, 2020 12:15 PM: Running Backup took 0:00:00:00.520
Oct 11, 2020 12:15 PM: ExecuteNonQuery: PRAGMA optimize took 0:00:00:00.000
Oct 11, 2020 12:15 PM: Starting - ExecuteNonQuery: PRAGMA optimize
Oct 11, 2020 12:15 PM: ExecuteNonQuery: DELETE FROM "RemoteOperation" WHERE "Timestamp" < 1599819300 took 0:00:00:00.000
Oct 11, 2020 12:15 PM: Starting - ExecuteNonQuery: DELETE FROM "RemoteOperation" WHERE "Timestamp" < 1599819300
Oct 11, 2020 12:15 PM: ExecuteNonQuery: DELETE FROM "LogData" WHERE "Timestamp" < 1599819300 took 0:00:00:00.000
Oct 11, 2020 12:15 PM: Starting - ExecuteNonQuery: DELETE FROM "LogData" WHERE "Timestamp" < 1599819300
Oct 11, 2020 12:15 PM: ExecuteNonQuery: UPDATE "RemoteVolume" SET "VerificationCount" = MAX(1, CASE WHEN "VerificationCount" <= 0 THEN (SELECT MAX("VerificationCount") FROM "RemoteVolume") ELSE "VerificationCount" + 1 END) WHERE "Name" = "duplicati-b8014c49423d648f2a536caf0778c0469.dblock.zip" took 0:00:00:00.009

Run-script-before-required

exit code 0

  • Edit backup configuration, remove run-script-before, add run-script-before-required with C:\Duplicati\test_exit.bat. Change C:\Duplicati\test_exit.bat back to exit 0 
Last successful backup:
    Today at 12:35 PM (took 00:00:01)
    Run now 
Next scheduled run:
    Today at 12:45 PM
Source:
    463.76 KB
Backup:
    471.18 KB / 3 Versions
  • The back information recovered from its ‘zero state’ as I call it where it shows 0 storage and 0 versions. Everything works as expected.

Exit 1

  • Add file to destination folder and change test_exit.bat to exit 1.
    When the automatic backup time passes:
  • The backup state remains:
Last successful backup:
    Today at 12:35 PM (took 00:00:01)
    Run now 
Next scheduled run:
    Today at 12:55 PM
Source:
    463.76 KB
Backup:
    471.18 KB / 3 Versions
  • The Duplicati icon in the taskbar turns red with a cross (its error state).
  • An error message pops up in the webserver interface:
Error while running Test_Backup
The script "C:\Duplicati\test_exit.bat" returned with exit code 1
  • The stored log shows: Oct 11, 2020 12:45 PM: Failed while executing "Backup" with id: 5

This is as expected because run-script-before-required should always raise an error when the exit code is not 1. From my understanding you should be able to get exactly the same behaviour when using run-script-before with exit 5 or maybe another one, but I did not test this (I can if you want me to).

What I expect

I expect run-script-before to remain its backup state when it exits with exit 1 instead its backup information changes to 0 storage, 0 versions. This is not useful when I want to check during the weekend if my backup ran during the week when my laptop was attached to a dockingstation. For the rest it works as expected. I have not looked at the other backup codes.

Sorry for the long post. I hope I presented my process in a clear manner, but I am happy to change / add to it :slight_smile:

10

Oh yeah, this definitely seems like a bug. These values shouldn’t change to zero. I’ll try investigating.

Thank you for the detailed post!

11

If you are willing can you please open a new issue on the Github project page? You have great detail here, it can mostly be copied and pasted.

Thanks!

12

I haven’t been following closely, but is this slightly related?

13

The Github issue is also about what happens after an exit 1 return from run-script-before. Hence it should be more or less about the same piece of code. However, from my tests I do not observe the ‘fatal error’ that S-dn-Y talks about except if I would run run-script-before-required. So, I cannot confirm his observations.

14

Hey man,

That’s actually my bug report.
To notice my bug, you must use the --run-script-after.
The GUI doesn’t catch an error indeed, exiting with 1 in the before script, but when you parse the job result in the after script, the job results in fatal.

I can acknowledge this bug you’re describing though als on a docker based install as most of my jobs also have zero job statistics in the GUI when nothing changed in the structure (and the backup was cancelled with exit 1).

Now you may be partially right in that it’s related, but going through the source code a few months back I did not notice it. I can however imagine that the fact that the job parses the result as fatal under the hood, makes certain subtasks that are responsible for updating the job details not run, but you’d need to dive into the code to know for sure.

Cheers!

15

Confirmed… I can see the Parsed Result is set to Fatal if the before-backup script exits with 1, 3, or 5. I didn’t test other values.

The issue where this is shown:

Source:
    0 bytes
Backup:
    0 bytes / 0 Versions

Seems to be due to the metadata of the aborted backup results being stored in the database. I don’t think aborted/failed jobs should have their metadata stored.

16

FYI I did a test to see if I could persuade it to record empty statistics in a job log or email report. It wouldn’t.
Maybe if you get a feel for how results flow, failed jobs can someday go into the job logs not the server log.

17

Do you mean you weren’t able to reproduce the issue? I could reproduce with a run-script-before batch file that has just one line: exit 1

Can you try that out?

18

That makes sense, thanks!
Now we just need someone to code it in :rofl:

Exit 1 should not really count as a failed job should it though? As it’s an intended stop. Therefore the Fatal jobresult isn’t right, Fatal in my opinion means: ACTION NOW! :slightly_smiling_face: I would use Exit 5 if I wanted to trigger an error.

Cheers!

19

I did, reproduced home page metadata clobber with exactly that script, could not persuade it to misbehave in a similar way for two other results reporting methods (log and email), thus shooting down a theory that faked outcome with uninitialized (so zero) result statistics was flowing into all of the result-reporting spots.

To give an example from one of my backups, job log BackendStatistics “KnownFileSize” seems to match Backup size, “BackupListCount” matches Versions, “ExaminedFiles” (not in BackendStatistics section) matches Source size, dividing by 1024 * 1024 * 1024 in my case to get the units set to GB for the display.

Right, however intended stops happen multiple ways. I think the 1 says to stop and treat it as a success, however I think you know the plan because you say to use exit 5 for an error, and that fits the plan below:

For code research, the link above is also an easy path (via GitHub blame) to the code change adding this.

My comment had a more extended context than the current problem. There is a recurring problem where actual failed backups don’t record the failure in the job log, instead putting it in the server log, while giving the user a Show button in the error popup that takes them to the job log. Sometimes that’s the wrong spot.

The code research to keep the exit 1 from wiping the statistics seemed like potentially the understanding gained from that would be a step to solving the wrong-log bugs by collecting job errors in only the job log. Some other operations seem to do better, as do backup warnings, but backup errors seem to go astray.

20

Thanks to everyone for sharing their knowledge about this problem. Shall I make a concise Github issue regarding the run-script-before with exit 1 wiping the metadata/statistics and referring to this thread since it looks like that solving this issue results in partial understanding of other bugs (like logs and errors)?

EDIT:
I created a Github Issue: #4344