Which web domains do I need to whitelist to backup to onedrive?

roasted_watermelon · March 16, 2025, 8:11pm

I am using duplicati packaged to docker by linuxserver.io.

I am running duplicati on a restricted network where I have to whitelist each domain that duplicati needs access to. Basically each docker container needs to define a list of urls it needs to access. I am using onedrive to backup. Which domains do I need to whitelist?

Thanks.

roasted_watermelon · March 16, 2025, 9:25pm

Okay got it.

duplicati-oauth-handler.appspot.com
graph.microsoft.com

Had to look around the source code to figure it out.

Edit: Nevermind, spoke too soon. The backup gets stuck.

Edit 2:
After some network monitoring, added the following

onedrive.live.com
del11s20-in-f20.1e100.net

The first one is obvious. The second one is some google domain, which I don’t have any idea about.

kenkendk · March 17, 2025, 7:34am

Hi @roasted_watermelon, welcome to the forum

It looks like a Google thing. Based on that, I would guess that it is related to the OAuth service somehow, as that is hosted on Google infrastructure.

What I don’t understand is why this is visible to Duplicati, as it should just see the entry server on appspot.com. But if it is indeed something that is needed, it will likely fail in the future, as each server has a unique hostname, and depending on what host you hit, you will get a different hostname.

Could it be that this is related to some Google Home appliance and not Duplicati?

ts678 · March 17, 2025, 12:17pm

I think an app’s address is not static, which will interfere with whitelists.
Single IP of duplicati-oauth-handler.appspot.com changes (but slowly).

roasted_watermelon · March 17, 2025, 1:22pm

@kenkendk @ts678 I removed that del11s20-in-f20.1e100.net and it still works, so I guess even if it does change in the future, it is not a big deal.

kenkendk · March 17, 2025, 1:32pm

It is running on multiple machines at the same time to load-balance, but none of this should be visible to the caller.

ts678 · March 17, 2025, 2:07pm

It’s invisible to normal caller, but one who whitelists name or IP gets surprised, correct? I suppose if whitelist was updated by name at the DNS lookup before Duplicati’s use, it would probably work (because Duplicati will use it), but how’s precision updating like that possible? I’m not a restricted network whitelist guru.

One also can’t reverse-DNS the IP to see if name is allowed. It doesn’t reverse.

kenkendk · March 17, 2025, 3:06pm

The logic should be that the caller resolves duplicati-oauth-handler.appspot.com to whatever IP is currently serving requests. After that, the connection is made to that IP and that is the end of it.

But you have a point; if the client somehow tries to do a reverse hostname lookup from that IP, it would (presumably) end up getting something that looks like the 1e100.net address.

But I don’t know why any part of Duplicati would do a reverse hostname lookup.
Maybe the security software does this as an extra “service”?

ts678 · March 17, 2025, 5:24pm

Except in whitelist case, the connection may get blocked. That’s the concern.

Presumption is that some IP level blocker needs an IP whitelist, but what IP?
Today’s IP might not be tomorrow’s IP. It likely prefers static IP, but lacks that.
There might be a way to make it static with a custom domain, but I’m unsure.

Here’s what I mean by dynamic (rather than static). IP changes occasionally:

$ dig duplicati-oauth-handler.appspot.com

; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> duplicati-oauth-handler.appspot.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 52846
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;duplicati-oauth-handler.appspot.com. IN	A

;; ANSWER SECTION:
duplicati-oauth-handler.appspot.com. 300 IN A	172.253.62.153


$ dig duplicati-oauth-handler.appspot.com

; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> duplicati-oauth-handler.appspot.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 33126
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;duplicati-oauth-handler.appspot.com. IN	A

;; ANSWER SECTION:
duplicati-oauth-handler.appspot.com. 227 IN A	142.251.167.153


$ dig duplicati-oauth-handler.appspot.com

; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> duplicati-oauth-handler.appspot.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7483
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;duplicati-oauth-handler.appspot.com. IN	A

;; ANSWER SECTION:
duplicati-oauth-handler.appspot.com. 300 IN A	172.253.62.153


$ dig -x 172.253.62.153

; <<>> DiG 9.18.30-0ubuntu0.22.04.2-Ubuntu <<>> -x 172.253.62.153
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30787
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;153.62.253.172.in-addr.arpa.	IN	PTR

;; ANSWER SECTION:
153.62.253.172.in-addr.arpa. 2917 IN	PTR	bc-in-f153.1e100.net.


(OK, technically it reverses, but not into anything you'd whitelist)

If so, it doesn’t help because this (and most IPs I saw) won’t reverse lookup.

Maybe @roasted_watermelon can get some info on whitelisting dynamic IP?
Or if somehow the current setup works long-term, great, but I’d be puzzled…

kenkendk · March 17, 2025, 8:21pm

I think the logic here is to use hostname filtering, because IP filtering will not work.
Google is free to rotate the IPs used every 5 minutes, making it really tedious to maintain a whitelist on that level.

ts678 · March 17, 2025, 8:35pm

That sounds like it would work if the restricted network can do it.
Some systems apparently snoop on DNS requests to achieve it.
Not all DNS is easy, e.g. Firefox DNS over HTTPS may be hard.
OTOH some systems man-in-the-middle HTTPS traffic to check.

kenkendk · March 17, 2025, 8:50pm

For TLS conections, I would assume they use SNI to extract the hostname.
This works regardless of the DNS method.

roasted_watermelon · March 17, 2025, 9:02pm

@kenkendk this is the command that is used to unblock a domain:

sudo iptables -I DOCKER-USER -s "$container_ip" -d "$DOMAIN" -j ACCEPT

So it is not a specific IP. Docker somehow finds all the IPs needed to be unblocked from the domain name itself.

ts678 · March 17, 2025, 9:18pm

Excerpts from the iptables man page to see what -d does:

      [!] -d, --destination address[/mask][,...]
              Destination  specification.   See  the  description  of  the  -s
              (source)  flag  for  a  detailed description of the syntax.

      [!] -s, --source address[/mask][,...]
              Source specification. Address can be either a  network  name,  a
              hostname,  a  network IP address (with /mask), or a plain IP ad‐
              dress. Hostnames will be resolved once only, before the rule  is
              submitted  to  the kernel.  Please note that specifying any name
              to be resolved with a remote query such as DNS is a  really  bad
              idea.

Note the last line, which is probably related to earlier mention of “resolved once”.

Well, I guess we can stop discussing network boxes, and what skills they have…

roasted_watermelon · March 17, 2025, 9:52pm

Interesting… I guess I will continue observing and see…

ts678 · March 21, 2025, 8:44pm

The result will be interesting. If iptables gets refreshed very often, it might work.

I’m probably going to stop watching the IP of that host, but the results I found are:

$ while true; do
> date
> host -t A duplicati-oauth-handler.appspot.com
> sleep 3600
> done
Tue Mar 18 11:41:02 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Tue Mar 18 12:41:02 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Tue Mar 18 01:41:02 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Tue Mar 18 02:41:02 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Tue Mar 18 03:41:02 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 172.253.62.153
Tue Mar 18 04:41:02 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Tue Mar 18 05:41:03 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 172.253.62.153
Tue Mar 18 06:41:03 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Tue Mar 18 07:41:03 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Tue Mar 18 08:41:03 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Tue Mar 18 09:41:03 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Tue Mar 18 10:41:03 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Tue Mar 18 11:41:03 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Wed Mar 19 12:41:03 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Wed Mar 19 01:41:04 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Wed Mar 19 02:41:04 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Wed Mar 19 03:41:04 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Wed Mar 19 04:41:04 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Wed Mar 19 05:41:04 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Wed Mar 19 06:41:04 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Wed Mar 19 07:41:04 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Wed Mar 19 08:41:04 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Wed Mar 19 09:41:05 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Wed Mar 19 10:41:05 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Wed Mar 19 11:41:05 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Wed Mar 19 12:41:05 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Wed Mar 19 01:41:05 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Wed Mar 19 02:41:06 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Wed Mar 19 03:41:06 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Wed Mar 19 04:41:06 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Wed Mar 19 05:41:07 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Wed Mar 19 06:41:07 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Wed Mar 19 07:41:07 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Wed Mar 19 08:41:07 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Wed Mar 19 09:41:07 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Wed Mar 19 10:41:07 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Wed Mar 19 11:41:07 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Thu Mar 20 12:41:08 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Thu Mar 20 01:41:08 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Thu Mar 20 02:41:08 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Thu Mar 20 03:41:09 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Thu Mar 20 04:41:09 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Thu Mar 20 05:41:09 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Thu Mar 20 06:41:10 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Thu Mar 20 07:41:10 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Thu Mar 20 08:41:10 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Thu Mar 20 09:41:10 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Thu Mar 20 10:41:11 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Thu Mar 20 11:41:11 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Thu Mar 20 12:41:12 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Thu Mar 20 01:41:12 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Thu Mar 20 02:41:12 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Thu Mar 20 03:41:13 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Thu Mar 20 04:41:13 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Thu Mar 20 05:41:15 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Thu Mar 20 06:41:16 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Thu Mar 20 07:41:16 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Thu Mar 20 08:41:17 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Thu Mar 20 09:41:24 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Thu Mar 20 10:41:24 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Thu Mar 20 11:41:24 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Fri Mar 21 12:41:24 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Fri Mar 21 01:41:25 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Fri Mar 21 02:41:25 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Fri Mar 21 03:41:25 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Fri Mar 21 04:41:25 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Fri Mar 21 05:41:26 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Fri Mar 21 06:41:26 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Fri Mar 21 07:41:26 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Fri Mar 21 08:41:28 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Fri Mar 21 09:41:29 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Fri Mar 21 10:41:29 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.167.153
Fri Mar 21 11:41:29 AM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Fri Mar 21 12:41:31 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Fri Mar 21 01:41:32 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Fri Mar 21 02:41:32 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Fri Mar 21 03:41:34 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153
Fri Mar 21 04:41:39 PM EDT 2025
duplicati-oauth-handler.appspot.com has address 142.251.163.153