Announcing dupReport - A Duplicati Email Report Summary Generator


#21

Just to let everyone know, a new version of dupReport beta 2.0.3 has been posted to the beta branch on the dupReport project on GitHub. Lots of bug fixes and a few new options. Check it out if you can.

I’ll let it ‘bake in’ for a week or so and if there are no more problems I’ll merge it into the master branch sometime next week.

Enjoy!


#22

I have a suggestion for an alternative way of formatting reports. The basic idea is to create a visual grouping when the same source is backed up to multiple destinations.

A picture is worth a thousand words – this google sheet is a mockup of what I am thinking.

Food for thought…

Marc


#23

Hmmm. Interesting approach. It’s totally doable as the information is already in the database, and is basically just a variation of the way the report is generated now. I have a feature on the drawing board that would allow the user to select the columns displayed on the report. It may be time to re-think the way the reports are produced. Let me think about this for a bit…


#24

Fantastic idea – I love it!

Another idea for enabling report customization that I think would pair well with what you described: Allow the user to format the sub-header row that currently contains “”*** source-name to destination-name ***" with a template that allows keyword substitution. ie, Something like %SOURCENAME% would be substituted with the actual source name.

A new sub-head row would need to be generated anytime one of the keywords used in the template changes.

Marc


#25

Marc,

I coded up a rough demo version of what you’re talking about in the source/destination grouping, just as a proof-of-concept. It works, but it’s far from elegant. Or pretty. I still need to think about how to best handle the sub-header formatting.

Right now I’m trying to keep 2.0.3 stable for a week or so then I can merge it into the master branch as an official release. As soon as that’s done I can open up 2.0.4 for development and start on the reporting fixes (as well as some other goodies).

Stay tuned…


#26

Very cool! Let me know if you want me to do some testing…

Marc


#27

Hey folks!

dupReport 2.0.3 has been released on GitHub. Lots of bug fixes and a few extra features. Check out the changelog file for details. If you haven’t upgraded since the early days, you might want to grab this release.

I’m now turning my attention to 2.0.4, which will be mostly related to improvements in the reporting features of the program. A lot of it is based on some suggestions by Marc Aronson about different ways to sort the report, but I’m also looking at a couple of more features, lots of internal code improvement, and (gasp!) bug fixes. I’m also moving my development platform from Linux to Windows. The program is getting too big for nano editing and command-line git.

This one may take a while, but I’ll keep everyone updated on progress.

Enjoy!


#28

Hi - just downloaded dupReport 2.0.3 to give it a try.

The initialization step using dupReport.py -i causes an exception: NameError: name ‘write_log_message’ is not defined.

My assumption is that it should be ‘write_log_entry’ which seems to work fine.


#29

Not sure how that happened. I had a bit of trouble switching development platforms last night and it may have crept into there. Just uploaded fixed code to the master branch. Thanks for the notice.


#30

Due to popular demand (a couple of people asked), I have opened up a Twitter account for dupReport. Follow @dupreport for release announcements, support, and program info. Low volume but high quality :wink: .


#31

Hello HandyGuy, I made a small change, see if you find it interesting: a highlight in the lines with the names of the jobs. When you have a lot of small backups, I found the report look a bit confusing, so I put a background in the line with the names. Look:

new

I just put the color tag here (line 690):

msgHtml = msgHtml + '<td bgcolor="#E6E6E6" align="center" colspan = "11"><b>{}</b></td>'.format(txt[0])

#32

Interesting option. I’ll add it to the 2.1 features list.


#33

Hey, everyone: version 2.0.4 has been released and is now available on the GitHub ‘master’ branch. Some bug fixes in this release, including better handling for all sorts of date & time formats. Check it out!


#34

Hello everyone!

The first beta version of dupReport 2.1 has been posted to GitHub and is ready for download and testing. Lots of great additions to this release. A few of the highlights:

  • Added new reports to show backup jobs by source, destination, and run date
  • Date/time format can be specified per src/dest pair, if jobs are running in different locales.
  • Dates can now be displayed in 12- or 24-hour format
  • Report can now use keyword substitution for subheading titles
  • Reports can now be sent to one or more files in HTML, txt, or csv formats (-f command line option)
  • Can prevent sending report email if only saving to output file (-x command line option)
  • Option to email job warnings/errors if in collect mode (warnoncollect= .rc option)
  • Column headings can be customized ([headings] section in .rc file)
  • User can select which columns to display in a report ([headings] section in .rc file)
  • Can specify background colors for subtitles, messages, warnings, and errors
  • Database & .rc file upgrades now handled automatically
  • Ability to roll back database to a specific time/date (-b option). Useful for failed runs or testing.

Full details and documentation are on the GitHub page.

If you’re upgrading from a previous version, the new program should upgrade your .rc and database files automatically, but as a caution until the final release it’s advisable to back up these files to a safe location until you’re sure everything is working properly.

Comments, suggestions, and bug reports can be made here or in the Issues section on GutHub.

Enjoy!

HG


#35

Is it available in a Docker container yet? :grin:


#36

I’ll add it to the future development list :wink:


#37

Just wanted to let everyone know that the latest (and greatest) beta of dupReport 2.1 is now available on the GitHub site. Tons of new reporting features and bug fixes.

I’m going to let it burn in over the holidays and (assuming nobody finds any major problems) will release it sometime after the new year.

Enjoy!


#38

Trying out the 2.1 beta, using Office365 email account. I’m getting the following error:

C:\dupReport-2.1.0_Beta>python dupReport.py
Traceback (most recent call last):
File “dupReport.py”, line 209, in
newMessages = globs.inServer.checkForMessages()
File “C:\dupReport-2.1.0_Beta\dremail.py”, line 177, in checkForMessages
retVal, data = self.server.search(None, “ALL”)
File “C:\Users\xxx\AppData\Local\Programs\Python\Python36-32\lib\imaplib.py”, line 718, in search
typ, dat = self._simple_command(name, *criteria)
File “C:\Users\xxx\AppData\Local\Programs\Python\Python36-32\lib\imaplib.py”, line 1191, in _simple_command
return self._command_complete(name, self._command(name, *args))
File “C:\Users\xxx\AppData\Local\Programs\Python\Python36-32\lib\imaplib.py”, line 939, in _command
’, '.join(Commands[name])))
imaplib.error: command SEARCH illegal in state AUTH, only allowed in states SELECTED


#39

@ninja6o4, what logging level were you using? If not level 3, can you re-run with the -v3 option to get full debug output. The post the last 20 lines from the log file and I can try to see what’s happening.


#41

I see it - I had a rule to move backup emails to a subfolder, and dupReport wasn’t able to get to it - fixed it.
The log now shows it scanning through the emails in this folder, but it craps out part way through. the folder contains a mix of StorageCraft, UrBackup, and Duplicati automated emails.

Traceback (most recent call last):
File “dupReport.py”, line 213, in
globs.inServer.processMessage(nxtMsg)
File “C:\dupReport-2.1.0_Beta\dremail.py”, line 304, in processMessage
decode = email.header.decode_header(msg[‘Message-Id’])[0]
File “C:\Users\xxx\AppData\Local\Programs\Python\Python36-32\lib\email\header.py”, line 80, in decode_header
if not ecre.search(header):
TypeError: expected string or bytes-like object

I pruned the log, here is the last entry on it: ------=_NextPart_000_0025_01D1E182.8F2771A0-- ] messageId=[<002401d1e1bd$3b8 - Pastebin.com