Own updates server

luisBSBDATA · November 5, 2018, 2:43pm

Hi,

I would like to create my own updates server. I reviewed the script that you say (build-release.sh), and I understand that you compile the sln file and copy the files to aplhavss, you upload this files on aws, (I supose that your website are hosted by amazaon), and finaly you upload it on github.

I compile the binaries with Visual Studio and with the scripts on the folder /Installer/Windows I create the msi.(and it works) The build-release make the steps automacly to create de zip.

What’s the function of the manifest? It’s possible to create a webserver simple? I understand that your updates website only is visible for the duplicati because it have the key. Is posible delete this key? I don’t care if my website sample the directories. what I need to my webserver, the manifest?

I don’t know if is more easy create a webservice without manifest and without key or with these. But how duplicati find if exist a new update or not?

I works with version duplicati 2.0.3.11.

Any idea?

Thanks for your time.

JonMikelV · November 5, 2018, 4:51pm

Are you trying to create your own updates server to control what updates are offered to your Duplicati install base?

I’m not sure this has been done anywhere before.

luisBSBDATA · November 5, 2018, 5:25pm

I would like to offer my own custom versions of duplicati on my own updates server .

drwtsn32 · November 5, 2018, 6:57pm

Regarding the key - I assume that’s the public key for validating the download is authentic/unmodified. If you are already customizing and compiling Duplicati yourself, I don’t see why you couldn’t just remove this check. Or if you have your own code signing cert you could put your own public key in that file and sign the binaries yourself.

Looking at the code I also see that there is already some provision for an “OEM” to use their own update server, etc. Not sure how that is handled though.

luisBSBDATA · November 6, 2018, 8:50am

hi,

@drwtsn32 I see the file of OEM, but I think that is only for link to the url of the servers. How to created a new public key for my customizes?

I need information (how to and what do I need) to create the server.

mikaelmello · November 6, 2018, 11:13am

I am doing exactly that.

Although I develop on Windows, I found it easier to setup a WSL and use build-release.sh when launching releases.

I’ve modified the build-release.sh to upload the files to our own server and our version of Duplicati checks for that server instead of Duplicati’s.

When the files are uploaded to my server, I grab the zip and manually build the Windows installer (on Windows, not WSL) using build-msi.bat

drwtsn32 · November 6, 2018, 3:00pm

You would need to get a code signing certificate. The public key is part of that certificate.
Of just remove the checks entirely from the source code if you want to accept the additional risk, which may be minimal if this is an internal web server.

luisBSBDATA · November 6, 2018, 3:31pm

Hi,

Thanks @drwtsn32

@mikaelmello :
I develop on Windows too, but I use visual studio to compile the custom binaries. Actualy I created custom versions for windows, linux, macos, qnap and synology.

With the github console (Git bash) it is possible to execute the bash scripts on windows.

I see the build-release.sh, but in Duplicati they are using an AWS server and at the same time they upload it to github. But, are you using a simple web server or do you need a specific web server that simulate aws? You can explain more about this please?, I am lost.

Thanks.

mikaelmello · November 7, 2018, 8:20pm

I removed the code that uploaded to AWS and added something that uploads to our server that is not hosted by AWS

kenkendk · November 7, 2018, 9:35pm

The manifest contains a description of the update (name, changelog, zipfile url, and signing key)

In principle yes (you can change the code to remove the check), but I suggest that you just create your own key. I have set it up such that you can avoid changing the repo source code, and just place the key outside the source code folder:

https://github.com/duplicati/duplicati/wiki/Branding-and-OEM-customization#handling-updates

Otherwise you essentially allow anyone with sufficient access to deploy malicious updates (actually any binary they like) to all your clients.

Yes. Duplicati’s updater checks the urls for the latest.manifest, which is a file in json format, with a signature prepended to the file. If the version number is greater than the currently running version, it will display an update notice.

I just use S3 to have a persistent online storage. There is nothing in the updater that requires any particular service provider. After the files are uploaded, they are accessed by the Duplicati clients over plain HTTP(s). In fact, the updater does not read directly from S3 as the bandwidth costs are ridiculous, so all updates are mirrored and served from a proxy server not related to S3 or AWS.

You can even have the manifests and zip files on different servers (different urls) if you like. I just prefer to keep it in the same place.

Awesome!

luisBSBDATA · November 13, 2018, 5:18pm

Hi,

I will reseach how to use ftp protocol with bash to upload my files on the web server nad how to get a code signing certificate.

Thanks to all for your comments.

ts678 · November 13, 2018, 7:11pm

I suggest you use something other than ftp protocol if going over the Internet. FTP has no encryption for either your data or your login credentials. There are secure variations of the FTP name (e.g. SFTP or FTPS) though. You’ll probably find out more as you research, and ultimately it depends on what type of access your site gives.

kenkendk · November 13, 2018, 7:36pm

The signing is done with a plain RSA key. You can use the AutoUpdateBuilder tool to create a key:

github.com

duplicati/duplicati/blob/master/BuildTools/AutoUpdateBuilder/Program.cs#L61


      
          			var usedoptions = new [] { "allow-new-key", "input", "output", "keyfile", "manifest", "keyfile-password" };
          
                      if (string.IsNullOrWhiteSpace(inputfolder))
                      {
                          Console.WriteLine("Missing input folder");
                          return 4;
                      }
          
                      if (string.IsNullOrWhiteSpace(outputfolder))
                      {
                          Console.WriteLine("Missing output folder");
                          return 4;
                      }
          
                      if (string.IsNullOrWhiteSpace(keyfile))
                      {
                          Console.WriteLine("Missing keyfile");
                          return 4;
                      }
          
                      if (!System.IO.Directory.Exists(inputfolder))

You also use the AutoUpdateBuilder tool to generate the manifest, and optionally sign with your GPG key.

mikaelmello · November 14, 2018, 10:16am

Here is the process:

Generate a key using AutoUpdateBuilder, it will be encrypted and saved on the path you specified
Find a way to export the public part of the key to XML and save it to both Updates/release_key.txt and Duplicati/Library/AutoUpdater/AutoUpdateSignKey

Find a way to export the public part of the key to XML

I did it by running build-release.sh after the new key was created, this way you get an error message saying that the public key found on Duplicati/Library/AutoUpdater/AutoUpdateSignKey has value X and it should be Y. Then you simply edit the file and set its content to Y.

JonMikelV · November 14, 2018, 1:36pm

Thanks for the steps!

@luisBSBDATA, if this works for you are you OK the first post from updated to be a summary step-by-step guide so other users don’t have to dig through the whole thread?

luisBSBDATA · November 15, 2018, 4:17pm

Thanks for your help guides.

I can’t test it up to two or three weeks. When I come backy and test it, I will write here my results o my troubles.

I seem right

Thanks to all.

mmedin41 · January 30, 2019, 2:26pm

Hi, I have placed the oem-update-url.txt file in webroot and the oem folder but even though the Duplicati is updated, I need to do not update or show a new version of any idea or guide on how to do it.

To build use the build-msi.bat

Thank you.