Hey JonMike, I’m using it in the dupReport.py script. This is part of the DupReport.rc file which Marc posted about in this thread. I’m just trying to modify his wildcard a bit to capture all of the emails I already have in my mailbox from multiple PC backups. Does that clarify things?
There are some cases where a colon or a dash have a special meaning in a regex. Try escaping the characters in your regex (as in ‘\:’ or ‘\-’ but without the quotes) and see if that helps.
Thanks handyguy. I tried ^([\w ]*: |) and ^([\w ]*,: |).
I’m not sure what the comma is for after the asterisk, so I tried with and without it but neither worked. Sorry, this is my first time using Python. I can just change my email subject format to match what is already working so as not to waste anyone’s time.
Don’t give up so easily! I was able to use:
^\w+: Duplicati Backup report for
and it matched “Success: Duplicati Backup report for test1-test2”. The secret to parsing regex’s is to try to keep the expression as simple as possible. The above regex says the following:
- ^ - Match the beginning of the line
- \w - match any alphanumeric: [0-9a-zA-Z_]
- + - match one or more instances of the previous spec (i.e., alphanumeric)
- : - match the colon character immediately following the alphanumeric characters
- (space) - match a single space character after the colon. You can also use ‘\s’ to match any whitespace character (including tab, newline, etc.). If you anticipate multiple space characters you can use ‘\s+’ (specifying one or more instances of \s).
- Duplicati Backup report for… - match the literal string
I think the parentheses and the pipe character (‘|’) in your original regex were throwing your match off, as each has it’s own special meaning in regex’s.
There’s a good regex parsing page at https://pythex.org/ that lets you put in your regular expression and test strings to see if they match.
I can just change my email subject format to match what is already working so as not to waste anyone’s time.
Make the technology work the way you want, not the other way around! 
Hope this helps.
HG
I cannot strongly enough recommend people not use their primary gmail account for this, as the password is stored in the clear unless you encrypt the duplicati config. Set up a dedicated account for sending the email from.
Thanks a lot for the Pythex suggestion. That is exactly what I needed to do my testing. I’m just way over my head on this stuff, and I don’t like asking someone to spell it out for me due to the time it takes, but thanks for doing that anyway.
Unfortunately, the software does not seem to agree with Pythex. I have tried many different expressions, which match in Pythex but produce an empty table when used in dupReport.rc. Here are a few examples below that match in Pythex, but don’t work in the RC file.
- subjectregex = ^\w+: Duplicati Backup report
- subjectregex = ^\w+:\sDuplicati Backup report
- subjectregex = ^(\w+: |)Duplicati Backup report
- subjectregex = ^\w+:|Duplicati Backup report
- subjectregex = Duplicati Backup report
I don’t really see why matching the beginning of the line is required in this case. I feel like I should just be able to search the string “Duplicati Backup report”. Again, works in Pythex, not in the RC. Am I doing something wrong? My exact email subject is as you stated. “Success: Duplicati Backup report for test1-test2” without the quotes.
In Duplicati I’m using this string.
- –send-mail-subject=%PARSEDRESULT%: Duplicati %OPERATIONNAME% report for %backup-name%
Also, FYI, the dupReport.rc file had two “outsender” settings in the [outgoing] section RC file. I removed one and it seems to work ok.
@NotHere. I agree. I do wish there was an easy way to encrypt it. I did make a dedicated gmail account and also used an application specific password, so it can’t be used to log into the account.
Although I have no answer for dash, colon fails might be extractHeaders code taking “Success:” as a header, which will cause the regex to fail because it expects that to be part of the string following a “Subject:” header. Another guess is that the configuration parser which is documented as taking a colon or equals, is confused.
Basically, the Pythex exam is useful, but the regex and the subject line both have to get there as they should.
Thanks, I re-tried with the dash instead of a colon and it works just fine. With this method I can just use
- subjectregex = Duplicati Backup report
and it works great. It definitely doesn’t like the colon for some reason.
EDIT - Note I originally thought this worked, but later realized using the dash broke the source/destination portion. Continue reading below for a fix.
To figure out which of the two reasons it is, you could try adjusting Duplicati and subjectregex to have a space before the colon. I think that will be enough to avoid mistaken identity. Header parser needs a colon attached.
Putting the space before the colon works using the same regex as my last message. I’m sure it would work with the wildcards as well, but it seems unnecessary to me since I’ll never have other emails with those words in the subject in this mailbox.
I think that’s evidence that Duplicati information after the email Subject: string was missing before. Thanks. Maybe dupReport will someday recognize “header” names only at start of the line. Multiline mode might help. Meanwhile, I’m glad you found a workaround that is enough for your situation and for others doing the same.
Thanks for the quick help everyone!
Glad to hear you got it working (mostly) like you want!
Just to be clear, this is what worked for you?
- In Duplicati:
–-send-mail-subject=%PARSEDRESULT% - Duplicati %OPERATIONNAME% report for %backup-name%(Note the dash, not a colon, after %PARSEDRESULT% - In dupReport.py script (of DupReport.rc file):
subjectregex = Duplicati Backup report
That is correct! Only thing to note is the --send-mail-subject needs the 2 dashes at the front in Duplicati, not just one. I’m sure that was just a typo though.
I believe it would also work if you did not put spaces, like %PARSEDRESULT%-Duplicati. I definitely tried %PARSEDRESULT%- Duplicati, which worked with the same regex, as did using a comma instead of a dash.
EDIT - Note I originally thought this worked, but later realized using the dash broke the source/destination portion. Continue reading below for a fix.
1 Like
Hi folks. Sorry I’ve been absent from the conversation, I took a few days off for the holiday.
After a small bit of testing I can confirm @ts678’s theory that the colon in the subject specification is confusing extractHeaders() and it’s seeing “Success:” as a separate header field. This causes the program to parse the header info as follows:
Clearly, ‘Success’ is not a standard email header. DOH!
I will open an issue on the GitHub site for this and see if there is a better way to parse the headers to avoid this problem. Stay tuned…
HG
2 Likes
OK, I think I have a fix for this. I rewrote and simplified the extractHeaders() code to address the “double colon” problem. This will allow colons in subject lines like “Success: Backup for test1-test2: All is well”.
New code has been uploaded to the Issue_103 Branch on GitHub. Please download that code and try it to see if it (1) fixes the colon issue and (2) doesn’t break already-existing installations (if you are using “normal” subject lines.)
I may also have an explanation for the dash (-) issue described above. dR uses the dash as the default source/destination separator (as in <source>-<destination>). If there is an additional dash in the subject line that may throw things off, depending on how it’s used. If you’re really intent on using a dash in the subject and you find it’s not working for you, change the [main] “srcdestdelimiter =” option in the .rc file to some other character, then make sure your backup job names use the new character. Please don’t ask me to provide a similar fix for this “dash” problem, as “<source>-<destination>” is used all over the code base and is not as isolated as the subject colon issue was.
Anxious to see if this gets it right. Please let me know.
HG
JonMike, see handyguy’s comments below. I thought it was working with the dash, but didn’t look closely enough that all of my source/destinations had been stripped from the table, so all backups were being put together in the table as a single blank src/dst.
handyguy, thanks for taking the time to work on this. I can confirm the src/dst did break when I used the dash (I didn’t notice until just today). I can also confirm I downloaded Issue_103 code and using success: is now working for me!
I haven’t fully tested failure: and warning: though. I had one old warning report. The word warning did show up in the table, but the red/yellow formatting and error description isn’t there. I haven’t read up on this section of the guide though, so not sure when the added detail is triggered (maybe only on more recent warnings?). I’ll post back once I have a more thorough test.
Looks like the red/yellow formatting and details only show up for failure reports, not successful reports with warning. If this is the normal behavior, then I just confirmed this is working. Thanks again!