Announcing dupReport - A Duplicati Email Report Summary Generator

#82

@peoman

Some good news and some bad news. The good news is that your report led to finding a whole lot of bugs in some sadly-neglected parts of the POP3 handling code, including the crash error that you initially reported. I believe all of those have now been fixed and uploaded to the Issue_105 branch on Github. Please download the code from this branch and see if it fixes your crashing issue.

The bad news is that the lack of line items in your report doesn’t seem to be a bug, rather it’s a bad artifact of using POP3. POP3 seems to work slightly differently on different implementations, but in general it automatically marks any emails that are read as “seen” and only lets you download them once. After that, it won’t let you download them again even if they are still there on the server. That’s why you were able to generate a report the first time but it came up empty after that. It also means that if you have a client that reads mail from your inbox before dR reads them, dR will never see any new mail because your client has already had them marked as read.

One way to get around this might be to use a “Leave messages on the server” option. This option tells the server to leave the emails marked as unread even if they are downloaded to a client. Support for this and the method for enabling it seems to vary from system to system, but instructions for doing this on Exchange/Outlook are here. If you’re not using Outlook hopefully this will give you enough information to get it working on your client.

I am using Gmail as my server and was able to reproduce both the conditions resulting an empty report (what you originally reported) and a successful run using the “Leave on server” process. Gmail allows you to set this on the server, whereas I believe Exchange requires you to set this on the client (make sure you set it for all the clients you use for this account, or it won’t work). As an added evil twist, Gmail also requires you to put a “recent:” tag in front of your user name in order to fetch the most recent emails in your inbox instead of the oldest. (Like I said, POP3 implementations vary from system to system.) I don’t think you’ll have that issue on Exchange, but since I don’t have an Exchange server to test with I wasn’t able to reproduce/verify that piece.

Anyway, please download the Issue_105 code and see if you can get it working on your system. If it checks out I’ll post it to pre_prod for others to test (there are some other enhancements I included as well). Let me know how it goes. To keep this thread uncluttered, please post any bug reports for this on the Issue_108 thread on GitHub. (My numbering scheme needs some improvement! :slight_smile: )

Good luck,

HG

#83

Hello all,

I’m happy to report that dupReport Version 2.2.5 is available in the pre_prod branch on GitHub. In addition to fixing some latent POP3 bugs, it adds the ability to mask sensitive data (by default) such as user name, password, email server, etc in the log files. This comes in handy if you have to send the log files somewhere else. Check out the README.md in 2.2.5 for more details.

1 Like
#84

2.2.5 has now been released to the master branch. Enjoy! :slight_smile:

#85

Hi, thank-you for creating this, definitely missed its information when I moved away from CrashPlan. I have it running nice on a small Fedora box I have for such things.

One question: can it mark the emails as “read” once processed? I use IMAP on my own mail server and can see the dedicated mailbox on my normal client, but all the emails remain unread. So unless I missed a setting it would be nice if dupReport marked each as read once it had processed them. Also simple way to know that all is still well.

#86

@Taomyn, the program takes a “leave no trace” approach when accessing mailboxes, as I can’t be sure what other programs may be using the same mailbox and I don’t want to interfere with their operation. However, I could probably add an option to allow for marking emails as read once dR sees them. Not sure about POP3 but pretty sure it can be done for IMAP.

Please open up an issue on the GitHub site and I’ll take a look into it. It may be a couple of weeks before I can get to it, as I have a few other things piled up in front of it.

HG

1 Like
#87

Issue opened, thanks.

#88

Hey, all. dupReport veriosn 2.2.6 is now available for testing on the pre_prod branch on GitHub. This release incorporates @Taomyn’s suggestion of enabling the ability to mark messages as read/seen once dupReport has finished processing them. Please download and take it for a test spin if you are so inclined.

Thnks,

HG

1 Like
#89

Is there any similar tool to do the same task offline.

Instead of reading emails and generating a report… Read the backup sqlite file and gererate reports…

Would this be a nice Idea

#90

Problem with that is the script would need access to every single database for each individual job on each individual machine you’re running Duplicati on. The emails being sent are generally all in a central location.

#91

Having received no objections, dupReport 2.2.6 has been released to the Master branch on GitHub. Enjoy, everyone!

HG