My best suggestion (assuming you’re under Windows hence the drive letters) is to go into Disk Manager and dedicate a drive letter to the particular drive you’re using for this backup. You can dedicate a higher letter that isn’t as likely to be subject to dynamic drive lettering as you add and remove removable drives.
You could also try using --alternate-destination-marker (only the drive with the text file having the right name will be used) and --alternate-target-paths (only look at these paths for the destination).
For more detail on the issue, see this:
I can’t find my notes on it right now, but there’s also a possible way to use --run-script-before with WMI drive enumeration which would let you grab a drive letter for a specific volume name then POSSIBLY pass that into the backup job as the destination. I hadn’t played with it beyond theory, though, so I’m not positive it would work.