Very unusual behaviour trying to use Duck Duck Go. Any suggestions for what to do?
Solution
I added 20.43.161.105 duckduckgo.com to my hosts file and everything is working fine now.
I also changed DNS servers away from my ISPs, thanks to all the recommendations in this thread.
cat /etc/resolve.conf
nameserver 1.1.1.1
nameserver 1.0.0.1
That seems to be working
> nslookup duckduckgo.com
Server: 1.1.1.1
Address: 1.1.1.1#53
Non-authoritative answer:
Name: duckduckgo.com
Address: 202.39.62.156
Name: duckduckgo.com
Address: 2001:b000:1a0:3505:202:39:62:15d
except (note that non-autoratative IP address which belongs to my ISP) ...
> ping -4 duckduckgo.com
PING duckduckgo.com (202.39.62.156) 56(84) bytes of data.
My ISPs address again. More...
traceroute to duckduckgo.com (202.39.62.156), 30 hops max, 60 byte packets
1 * * *
2 * * *
3 * * *
...
30 * * * *
Why do ping and traceroute not use the new DNS server's I've configured (after re-booting too_)
The only thing to work is to add
20.43.161.105 duckduckgo.com
to my hosts file and now everything seems to be working as expected, though I have doubts now that changing the DNS configuration has done any good.
I know ISPs cache things like youtube to reduce costs so I'm wondering if 202.39.62.156 handled caching of duckduckgo, and they pointed their nameservers there and that box is broken.
Thanks for everyone's input and patience (lol are you still reading???)
Original Question
I've used Duck Duck Go as my main search engine for many, many years.
I have several search engines installed in Firefox including 2 for duck duck go. One for the /lite version and one for the full version.
[See update at bottom]
In recent days neither of these work. I would type my query into the search engine, press enter as I have done for years.
All I see is a blank page.
The latest development is that when I try and enter ANY search to either of those engines I get a GOOGLE 404 not found page.
traceroute duckduckgo.com ─╯
traceroute to duckduckgo.com (216.239.38.120), 30 hops max, 60 byte packets
1 * * *
2 * * *
3 The usual internal routing of my ISP
4 "" "" ""
5 "" "" ""
6 "" "" ""
7 "" "" ""
8 * * *
9 any-in-2678.1e100.net (216.239.38.120) 4.089 ms 4.077 ms 4.181 ms
ping duckduckgo.com ─╯
PING duckduckgo.com (2001:4860:4802:32::78) 56 data bytes
64 bytes from any-in-2001-4860-4802-32--78.1e100.net (2001:4860:4802:32::78): icmp_seq=1 ttl=117 time=10.1 ms
64 bytes from any-in-2001-4860-4802-32--78.1e100.net (2001:4860:4802:32::78): icmp_seq=2 ttl=117 time=8.52 ms
64 bytes from any-in-2001-4860-4802-32--78.1e100.net (2001:4860:4802:32::78): icmp_seq=3 ttl=117 time=6.87 ms
64 bytes from any-in-2001-4860-4802-32--78.1e100.net (2001:4860:4802:32::78): icmp_seq=4 ttl=117 time=8.83 ms
--- duckduckgo.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 6.873/8.584/10.118/1.155 ms
cat /etc/resolv.conf
> MY ISPs name servers
> MY ISPs name servers
Sure enough I cannot find any pages on the site 2001-4860-4802-32--78.1e100.net which is obviously belongs to google.
This is very very strange.
Could someone verify if they can use DDG or whether they see the same as me?
Does anyone have any idea what's happening?
UPDATE
I can connect to and use DuckDuckGo using a browser VPN. This appears to be a mess made by my ISP.
I'd still like suggestion to overcome the problem though.
I hate to suggest this, but it very often* happens to me to forget this simple step: have you tried restarting? Restart firefox, then machine, then networking devices.
*it doesn’t actually happen that often because I rarely have iasues, which is why i sometimes forget to take the obvious simple steps.
I've re-booted my PC but hadn't thought about turning off the router. Definitely worth a try, thanks.
Given that the traceroute I ran listed 9 hops I think it's the ISP's server that's cached the wrong value.
It's a bit odd that you're getting an IPv6 address for DDG on the ping command, in my opinion. I assume something somewhere is not working on your setup with that.
You could try disabling IPv6 on your router.
Have you tried
digwith a different DNS server to see responses?With Google DNS:
And with Cloudflare:
If you get different responses, you could maybe try switching to the other one?
I agree.
216.239.38.120 is google.
Interestingly using dig with my ISP's DNS servers gives the same IP address as I get with the google and cloudflare servers.
Since only the VPN has worked I guess they are ignoring other DNS requests and it's somehow wrongly using a google address.
Using both cloudflare and google's give the same IP address for duck duck go: 20.43.161.105
Note that it is oddly different from the one posted by @Protected
I visit https://20.43.161.105 in my browser and I get an SSL warning:
Warning: Potential Security Risk Ahead - I usually see these when a website forgets to renew their SSL certificate. Clicking on the Advanced button then displays:
Weird.
When I view the certificate it looks like a valid certificate for DDG issued by DigiCert Inc.
Interestingly is says:
Validity
Not Before: Mon, 28 Oct 2024 00:00:00 GMT
Not After: Tue, 25 Nov 2025 23:59:59 GMT
Which is suspiciously around the time I started having issues.
Another button appears after a short delay saying Accept the Risk and Continue.
After clicking THAT links I then see google 404 not found page again!
.
Just guessing, but the SSL cert issue could be just because you are accessing the site via the ip and not the domain the certificate is issued for.
Also unless you are in the Netherlands it isn't that surprising you'd resolve different IPs for the same domain. I assume DDG has servers in multiple regions in the world and you just get the "closest" one.
/u/archevel is correct on both counts. The whole point of dns is that ip addresses are ephemeral. In the modern world, dns is used to localize people to the nearest server. You do not want to be using a DDG server from the USA normally. I have a personal server in the US, and I am in France right now. General interaction with that server is always annoying just because of the light speed lag to get across the Atlantic. It is possible to have a single IP address that goes to the nearest server (1.1.1.1, 8.8.8.8, and most other global DNS servers do this), but it takes BGP shenanigans and doesn’t make sense until you get to the scale of Cloudflare, Google, or other CDN.
The certificate issue is also exactly what archevel supposed. The certificate is perfectly valid for https://duckduckgo.com but not for https://ipaddress. It would be extremely weird for DDG to also issue their certificates for their ip addresses as well. It would be more unusual if you didn’t get an error accessing by IP.
My server runs its own dns. Both from it (Netherlands) and from Portugal using google's dns, duckduckgo.com resolves to 52.142.124.215 and traffic is (also) carried by Microsoft.
You could change your name servers, stop using your ISP's? Always good policy.
Thank you for that.
I've tried using both google(8.8.8.8) and cloudflare(1.1.1.1) DNS server both of which resolve duckduckgo.com to 20.43.161.105 which is different to the one you have???
See my reply to password1 for more strange behaviour.
Similarly using the IP address you posted directly in my browser (https://52.142.124.215 ) presents as duckduckgo.com in the address bar but displays the google 404 page???
Both IP addresses are valid for DDG. I'm sure they have a metric ton of IPs that their domains resolve to depending on various factors (primarily region; try your lookup again over a distant VPN and see if you get a different result).
Whenever I setup new ISP service, the first thing I do is always to get off the ISP's DNS. That's one small, but important, step in improving your privacy and security posture online.
Do you get the same weird result with a different browser in the same machine, and/or Firefox in private mode? If so, it might be an extension messing with the results.
Regardless of whether it fixes this problem, I second @Protected 's suggestion to change your DNS to not use your ISP's. You should be able to make this change in your router config so that it will apply to all your devices when they are on your home network.
Could also be a browser cache issue, which this test also helps with. If it does work in a private tab or another browser (or using
curl https://duckduckgo.com) then I’d try clearing your browser cache for DuckDuckGo.com and retrying.It's the same with Chromium. Tried private window, trouble-shooting mode, even a new profile.
Since posting and suggestions here I've become convinced it's the DNS cache at my service provider and that they're ignoring other DNS requests. It's all going through their modem/router where I don't have admin privileges.
I did read that most SOHO level routers have their own DNS server acting as a cache. Presumably they can't decrypt VPN traffic, but they can hijack/ignore DNS requests. Or the router cache is overly aggressive perhaps?
I'll give it another day or two and if it doesn't self-correct I'll contact them.
Weird that it's a single site (that I've discovered so far) and co-incident with DDG's new SSL certificate.
Honestly my recommendation is to bypass your ISPs DNS altogether. In all my life I have never seen an ISP DNS server that was not repeatedly serving me the wrong information or was unreasonably slow to resolve requests. Both Cloudflare and Google offer free public DNS servers that are very fast and up-to-date and there are other third parties that you can pay for if you’d like some extra features or value your privacy.
Don't use Google DNS, it's just a vector for them to parse your web traffic across the entire internet. Use Mullvad DNS for privacy, Cloudflare for raw speed, depending on your priorities. Mullvad also offers a variant with DNS-level adblocking which is especially useful on mobile devices.
Potentially something to do with this?
What I suspect is more likely is some kind of IP-based "shadowban" or rate limiting, most likely because some system/org in the chain of delivery has determined your IP address to be abusive - namely if you are behind CGNAT or have been doing any kind of web scraping or automation that flagged a detection system. Contact your ISP and request a unique IP if you don't have one already.