BSOD During Automatic Backup

smoothiebrain · February 15, 2025, 5:11am

Hello everyone,

I need help trying to figure out how to solve Duplicati causing BSOD. Stop Code reads, “Page Fault in Nonpaged Area”. I know Duplicati has something to do with this because the minidump file references “Duplicati.GUI.TrayIcon.exe”. (If you wanna see the .dmp file, let me know.) It also mentions “Wdfilter.sys”, which is apparently a Windows Defender file.

I have tried:

Reinstalling Windows Defender/Security
Reinstalling Duplicati (including using Revo Uninstaller to remove residual files and make a “clean install”)
Using " sfc /scannow" and “DISM /Online /Cleanup-Image /ScanHealth” / “DISM /Online /Cleanup-Image /RestoreHealth”
Tested my memory using memtest86
Rolled back to 2.1.0.3 (issue first started with 2.1.0.4)

EDIT: I should add the following info:

OS: Windows 10 22H2
Duplicati ver.: 2.1.0.4

Also added the fact I tried rolling back versions.

kenkendk · February 15, 2025, 10:24am

Searching for “wdfilter.sys bsod” gives a lot of results, so I am inclined to think this is the cause.
Duplicati itself does not have kernel access, so it should not be able to cause a BSOD.

I found this post on a microsoft forum that mentions some of the same.

Some suggestions in other places is to attempt temporarily to disable the real-time scanning feature in Windows Defender to see if that fixes the problem, but be sure you understand the risks before testing this.

Those versions are almost identical, only changes a timeout for WebDAV, so there should be no change in functionality otherwise.

What remote destination type are you using?

smoothiebrain · February 15, 2025, 4:20pm

Yes I scoured many articles and forum posts and tried all the things I mentioned above. Since then I’ve also tried running “chkdsk C: /f/x/r” to see if that does anything. I’ve seen links about updating display drivers but I don’t think that applies here. (Also I have an Nvidia card, and the latest drivers have apparently been a cluster.) Besides that, my only other option is going “scored earth” and reinstalling windows.

I understand the risk of turning off real-time protection protection off, hence why I haven’t done it. Also, even if that does fix the issue, what does that mean for me? Do I just leave the option off permanently?

Excuse my ignorance, but could you elaborate on what you mean by this?

I’m sorry for posting here, because honestly, I’m leaning towards this issue being the result of a Windows Defender AV update. However, I’m much more likely to get help and support from here or from the Duplicati devs on Github, than I am to get support and help from Microsoft.

smoothiebrain · February 18, 2025, 4:09am

I don’t consider this a permanent solution, but a workaround I found was to exclude the “Duplicati.GUI.TrayIcon.exe” process from Windows Security real time monitoring.

kenkendk · February 19, 2025, 10:22pm

At least that would make it clear where the problem lies. But yes, until a deeper fix is found, I guess it would need to be off.

I mean, where to you store the files? (S3, SSH, B2, FTP, etc). I was trying to see if it could be related to the backend implementation. There are some problems with Storj on Linux at least, so maybe this was something similar.

So it is a clash between something Duplicati does and Windows Defender.
Sounds like a decent compromise to only exclude one executable from the list.

smoothiebrain · February 21, 2025, 4:17am

Unfortunately, the BSOD came back as of tonight. Should I send the memory dump file as part of an issue on Github? If so, what other information would you like?

smoothiebrain · February 21, 2025, 5:03pm

Adding this for my own purpose. Things I have tried since I made the original post (including what I posted in the replies):

Ran “chkdsk” command
Added Duplicati as an exception from real time monitoring in Windows Defender/Security
Edited the registry to clear the page file upon shutdown (pending as of this post)

Things I’ve yet to do (as of this post):

Use driver verifier on wdfilter.sys specifically (not suitable as a permanent solution since it takes up a lot of CPU usage)
Reinstall windows

smoothiebrain · February 23, 2025, 12:02am

None of the above solutions have worked. Using Duplicati, whether through scheduled backup, or by running the backup job manually, has eventually resulted in a BSOD. Minidump files always point to a conflict between “wdfilter.sys” and Duplicati. The only solution left (besides changing backup software away from Duplicati) is to reinstall Windows.

Waiting for a response from Ken before I pull the trigger though.

ts678 · February 23, 2025, 12:38am

Is this something you can detail with an image or similar, so people can review?
I don’t personally do Windows internals, but perhaps something can be learned.

smoothiebrain · February 23, 2025, 3:31am

mini_dmp_files.zip (1006.7 KB)

There’s the last 5 minidump files.

Here’s the BSOD that pops up. I have several of these, but they’re all identical.

ts678 · February 23, 2025, 1:19pm

Asking again. Presumably you used some tool, and saw something indicating that.

Providing minidump files is great for those who do kernel debug, but that’s not me.

I don’t write Duplicati, but that’s a statement from somebody who does. Duplicati itself is an application program, written mostly to the higher-level .NET interface to allow portable code.

I think there are a few cases where a third-party .NET library dips into the Windows API, e.g.

https://github.com/alphaleonis/AlphaVSS

AlphaVSS is a .NET class library released under the MIT license providing a managed API for the Volume Shadow Copy Service also known as VSS.

and the developer website (which worryingly seems gone at the moment) says (via archive.org)

AlphaVSS is a .NET class library providing a managed API for the Volume Shadow Copy Service also known as VSS. The Volume Shadow Copy Service is a set of COM interfaces that implements a framework to allow volume backups to be performed while applications on a system continue to write to the volumes.

Component Object Model explains COM, and below that is likely (I don’t code this) Windows API, which also shouldn’t be able to BSOD AFAIK, or applications would always be crashing Windows.

What Duplicati version is this anyway? That determines what Microsoft runtime interprets its code. P.S. Never mind. That was given.

So there’s a really long chain between Duplicati and a BSOD, and I’d like to see output of the tool.

Meanwhile, I might try BlueScreenView on your minidumps. It’s easy, but doesn’t give much detail.

If I Google search 'BSOD" “PAGE_FAULT_IN_NONPAGED_AREA” it gives an AI summary FWIW.

Bug Check 0x50: PAGE_FAULT_IN_NONPAGED_AREA is the official page. You can read Cause.

Are you by any chance using VSS via the snapshot-policy option? If so, does turning it off help? Sorry about requesting tests that might crash the system. You can also test a small backup to see whether there’s something special about the BSOD backup that leads to the BSOD. You could run log-file=<path> log-file-log-level=verbose to see if it’s consistently around a certain place.

To minimize system crashes, you can start with the simplest possible backup until BSOD happens.

Using Event Viewer to see if anything interesting happened just before a BSOD may reveal things. Available references on this particular BSOD probably have lots of other ideas for troubleshooting.

ts678 · February 23, 2025, 1:46pm

As noted, BlueScreenView is pretty basic, but here’s what it has to say about your minidumps:

ts678 · February 23, 2025, 3:53pm

Analyzing Crash Dumps / Blue Screens (BSoD) suggests WinDbg as a next step. I installed from Microsoft Store as mentioned by WinDbg in Wikipedia. It’s easier, but I don’t know any downside.

WinDbg Crash Analyzer - Find Root Cause of Windows Blue Screen / Green Screen
helped me a bit with figuring out how to drive this. The way the UI worked seems unconventional.

PROCESS_NAME: Duplicati.GUI.TrayIcon.exe is there, but I don’t know how it derives that. A process being involved at time of BSOD doesn’t seem like it means the process did anything bad.

STACK_TEXT:  
ffffef8d`5e27eeb8 fffff801`79e45bcf     : 00000000`00000050 ffffef8d`5e280008 00000000`00000000 ffffef8d`5e27f160 : nt!KeBugCheckEx
ffffef8d`5e27eec0 fffff801`79c38cd0     : ffffba01`b63d6568 00000000`00000000 ffffef8d`5e27f1e0 00000000`00000000 : nt!MiSystemFault+0x1ce12f
ffffef8d`5e27efc0 fffff801`79e0e86d     : ffffef8d`5e27f1b0 fffff801`75575f7a ffffef8d`5e27f200 00000000`0000000d : nt!MmAccessFault+0x400
ffffef8d`5e27f160 fffff801`7a1233d0     : 00000000`00000000 00000000`00000038 fffff801`76273000 fffff801`755ab527 : nt!KiPageFault+0x36d
ffffef8d`5e27f2f0 fffff801`762b54ab     : ffffca88`6ef38570 00000000`00000000 00000000`00000000 00000000`00000000 : nt!FsRtlQueryCachedVdl+0x100
ffffef8d`5e27f3e0 fffff801`755764cb     : 00000000`00000000 00000000`00000000 ffffba01`00000000 ffffba01`b60132e0 : WdFilter!MpPreCleanup+0xf3b
ffffef8d`5e27f5e0 fffff801`75575f7a     : ffffef8d`5e27f800 00000000`00000012 ffffba01`d131d200 00000000`00000000 : FLTMGR!FltpPerformPreCallbacksWorker+0x36b
ffffef8d`5e27f700 fffff801`75575021     : ffffef8d`5e280000 ffffef8d`5e279000 ffffba01`cd6d1a20 ffffef8d`5e27f810 : FLTMGR!FltpPassThroughInternal+0xca
ffffef8d`5e27f750 fffff801`75574a2b     : ffffffff`fffe7960 00000000`00000000 00000000`00000000 ffffba01`c7899200 : FLTMGR!FltpPassThrough+0x541
ffffef8d`5e27f7e0 fffff801`79c4a295     : ffffba01`d131d2b0 fffff801`79c4a15d 00000000`00000000 00000000`00000000 : FLTMGR!FltpDispatch+0x8b
ffffef8d`5e27f840 fffff801`7a030277     : 00000000`00000001 ffffba01`d131d2b0 00000000`00000000 00000000`00043002 : nt!IofCallDriver+0x55
ffffef8d`5e27f880 fffff801`7a03831f     : ffffba01`c7899200 00000000`00000001 ffffca88`00000000 ffffba01`d131d280 : nt!IopCloseFile+0x177
ffffef8d`5e27f910 fffff801`7a03320c     : 00000000`00000f88 00000000`00000000 ffffba01`c093c080 fffff801`7a0f3910 : nt!ObCloseHandleTableEntry+0x51f
ffffef8d`5e27fa50 fffff801`79e12708     : ffffba01`c093c000 ffffba01`c8faffe0 ffffef8d`5e27fb40 ffffef8d`5e27fb40 : nt!NtClose+0xec
ffffef8d`5e27fac0 00007ff8`6cd0d6d4     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x28
00000013`7ab7ee78 00000000`00000000     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x00007ff8`6cd0d6d4


SYMBOL_NAME:  WdFilter!MpPreCleanup+f3b

MODULE_NAME: WdFilter

IMAGE_NAME:  WdFilter.sys

Looks (guessing a lot because I don’t do Windows internals) a file was closed, and Windows Defender did something it shouldn’t have done, thereby causing the BSOD. Specifics may be different than your screenshot, as I just picked one of the mindumps that you provided. It got:

PAGE_FAULT_IN_NONPAGED_AREA (50)
Invalid system memory was referenced.  This cannot be protected by try-except.
Typically the address is just plain bad or it is pointing at freed memory.
Arguments:
Arg1: ffffef8d5e280008, memory referenced.
Arg2: 0000000000000000, X64: bit 0 set if the fault was due to a not-present PTE.
	bit 1 is set if the fault was due to a write, clear if a read.
	bit 3 is set if the processor decided the fault was due to a corrupted PTE.
	bit 4 is set if the fault was due to attempted execute of a no-execute PTE.
	- ARM64: bit 1 is set if the fault was due to a write, clear if a read.
	bit 3 is set if the fault was due to attempted execute of a no-execute PTE.
Arg3: fffff8017a1233d0, If non-zero, the instruction address which referenced the bad memory
	address.
Arg4: 0000000000000002, (reserved)

The 2 in Arg4 is explained a little more in the web site documentation I cited for this BSOD:

0x2 - NONPAGED_BUGCHECK_NOT_PRESENT_PAGE_TABLE The address referenced does not have a valid active page table entry.

What that means should best be interpreted by a Windows internals expert. I don’t know if the Duplicati C# application developers happen to have such skills. You might need different help.

Sysinternals Process Monitor can look in detail at what Duplicati is doing. During a backup, it examines Source, possibly reading files. It will read and write temporary files. SQLite also will.

tempdir and asynchronous-upload-folder can move them, and Database could also be moved, which (unless it’s something I missed) would leave the Source area as candidate for scanning.

Using Process Monitor for something winding up in BSOD raises the question of whether it can write its information out in a way that will survive the BSOD. I don’t know. It does have ways to change its viewing position, as the view might vary above and below antivirus, due to “altitude”.

The Ultimate Guide to Procmon is a bit old, extensive, and one of many maybe-helpful articles.

EDIT:

Debugging Details:
------------------

*** WARNING: Check Image - Checksum mismatch - Dump: 0x2cb74d, File: 0x2cb743 - C:\ProgramData\Dbg\sym\win32kbase.sys\ABC321FD2d7000\win32kbase.sys

From dir command, it looks like WinDbg downloaded some information for me when I ran it:

 Directory of C:\ProgramData\Dbg\sym\win32kbase.sys

02/23/2025  09:17 AM    <DIR>          .
02/23/2025  09:17 AM    <DIR>          ..
02/23/2025  09:17 AM    <DIR>          ABC321FD2d7000

smoothiebrain · February 23, 2025, 6:13pm

First off, thanks for the extensive reply!

I’m not sure what that is, so if it’s not part of the default option, then I’m likely not using it.

I will definitely test that.

I can’t use that, as my backups are on a normal user account, and using procmon requires me to sign in to my admin account, which does not have the backups.

ts678 · February 23, 2025, 7:23pm

I don’t know for sure that Duplicati won’t go to AlphaVSS for other things, but I kind of doubt it.

It should give you a UAC prompt for elevation. It will run as elevated admin, but that should work.

Even an admin account would do UAC prompt, I think, but it would elevate as the same account.

If all else fails (but I doubt it will – at least not for that reason), you can do test backups as admin.

BTW the thought about moving things was meant to have a folder that’s excluded from Defender.

If Duplicati reading Source is blowing up Defender, excluding the whole area is probably too risky.

kenkendk · February 24, 2025, 9:33pm

Even if code calls the Windows API directly (.NET also does this), any application-level process should not be able to create a BSOD. The BSOD happens when something crashes at the kernel level (similar to Linux kernel panic), meaning there is a fault in either Windows or a kernel level driver.

For this issue, I am guessing that wdfilter.sys is running at kernel level and having a bug. This is somehow being triggered by Duplicati.

Excellent digging!

Seems like others are experiencing this too, and it looks like a new development:
https://answers.microsoft.com/en-us/windows/forum/all/blue-screen-of-death-caused-by-wdfiltersys/4ec05d47-e8bc-4410-a843-1944bfe9adbc

Yes, even if we did find the issue, it is highly likely that this cannot be fixed by Duplicati.

You need Administrator privileges to use it, so safe bet says you are not using it.

That might narrow it down. But it is possible that the file is not flushed due to the BSOD, so it will likely not point to the correct place, but somewhere near it.

You can also try to exclude folders manually and see if one of the folders are causing issues, and then drill into that folder.

smoothiebrain · February 25, 2025, 2:22am

Unfortunately, that is my post. After days of failing to finding a solution, I just posted on the Microsoft forums hoping for someone to offer something specific to try.

I’ve also noticed " WdFilter!MpPreCleanup" in the dump files, but there are no results about it online.

I reinstalled procmon, and it works now. I’ve tried using procexp as well to track down the process using the drive, and monitor it using procmon. Unfortunately, it seems the offending driver has become sentient, and has not shown up for me to monitor. Frustrating doesn’t begin to explain the intermittent nature of this issue.

EDIT: To add, since the process using wdfilter.sys didn’t show up, I was able to perform backups with no issue.

ts678 · February 25, 2025, 2:51am

“Changing Procmon’s Altitude (Capturing Lower-Level Events)” section in the article I mentioned might explain this. I haven’t tested it, but basically its view depends on location of the monitoring.

There are probably other articles around with hints on a suitable altitude, if that one doesn’t help.

I’m not sure if the driver is supposed to show up. From an image I found, it’s still original process.

Google search for “process monitor” “wdfilter” “altitude” images led me to an example with output:

How To: Run Process Monitor With Reduced Altitude

But since the crash has gone into hiding, it will be hard to find what did it in this way, and if it did crash, then the question would be whether File --> Backing files would catch the end of it.

smoothiebrain · February 26, 2025, 11:14pm

Turns out it was my display drivers causing this. M. Night Shyamalan has entered the chat…

After updating my display drivers, backups proceed with no BSOD. I’ve seen some posts about updating my display drivers but didn’t do it until now because the minidump files didn’t indicate anything about it. Also I had my display drivers for more than a month before it apparently became a problem.

On a side note, I couldn’t use the most recent drivers because it would result in a non-responsive black screen at login. I was lucky that there was one driver after the one that started all this but before the most recent one I could rollback to.

Anyway, I’m tired. I won’t mark this post as a solution for a few days, in case it does come back again and this was another red herring T_T