Ideas how to unlock Google's blocking of my YouTube RSS feeds
I subscribe to quite a few youtube channels to get notified when there are new videos posted. I've had this set up for several years.
Today I tried to add a new channel I've discovered. My RSS reader informs me it's blocked. I check all the other youtube feeds. Every single one of them reports "Error transferring <feed url>." server replied Forbidden (201).
Update: One day later and every feed is connecting and transferring again. It seems to be a temporary block. My IP address has changed overnight though so that's still my main suspicion.
It seems like I'm not the only victim
That contains a link to the author's issue on google's issue tracker
The official response is
Status: Won't Fix (Infeasible)
Unfortunately, there's nothing we could do here. Please reach out to community forum or Stackoverflow. Check out the link below:
They completely misunderstood the question - it's not asking how to find a feed, it's asking why that feed is getting blocked.
Not only this but using DuckDuckGo bangs for to search google get randomly sent to a captcha page - issuing the exact same query a second time goes through perfectly. The same is happening with keyword searches I set up in my browser. .
Any ideas what to do about this?
So sick of google's monopoly.
I can't really help, but just let you know that my experience has been similar. I'm using miniflux for RSS feeds, and sometimes my YT feeds also return a '403 forbidden', like reported in the issue tracker link.
I don't really have a solution, I just know that a few hours later it works again and I manually refresh a feed. There haven't been any issues for a long time now, but I don't know if that's a coincidence or not. I'm not using any VPN, it's just my home server fetching the RSS.
I hope it is temporary.
In the meantime I've exported my feeds, filtered out non youtube feeds and I'll some online feed readers. I'm expecting them to have google API keys and so I'll hope to hide behind them from Google's All Seeing Eye.
I'm starting with these 2 to see how it goes.
Are you using a VPN? I've noticed that Google sometimes blocks me from normal search if I'm using a VPN and in a private browser window, even though it will work fine through the VPN in a normal window, presumably because the lack of identity headers in combination with the IP address flags something in their systems. Since your RSS client presumably isn't sending any identifiable headers or cookies, that may be what you're hitting.
You might at least be able to narrow it down by seeing if the RSS link opens in your browser, and if there's any difference there between normal and private sessions? If that works, you can potentially narrow it down further by using curl or similar to send the same headers as your browser but the same user agent as your RSS client, to see if that's having any impact.
No - I'm not using a VPN.
There's no special network shenanigan at all, just a regular ADSL connection from an ISP. When I try and visit one of the feed urls in firefox I get a message
Google Sorry... We're sorry...... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.
See Google Help for more information.
Although that gives me an idea. I'll try from a fresh firefox profile to see if maybe my add-ins have triggered something. Probably have to wait until tomorrow and hope it's a temporary thing.
OK that's useful, it does suggest they've flagged your IP for one reason or another. Could be something on your side (maybe even as simple as your RSS client polling too often?), but it could also be that your ISP is using CGNAT and you're getting blacklisted for the actions of others on the network - or even mis-flagged simply because there are just too many normal users sharing that one IP address.
Ironically, the way to test that would be the exact opposite of my original question: if you do have access to a VPN you could fire it up and cycle through some different IP addresses with everything else held constant to see if it makes a difference. It's not perfect, because VPN IPs are significantly more likely to already be blacklisted than individual ones, but if you do manage to hit one that works it'd be another useful data point.
So I've just tried using Hoxx browser extension VPN and tried several youtube feeds which all of worked fine.
I'm going to try different IP addresses from my ISP to see if that helps.
I know youtube download scripts often rely on using browser cookies to avoid this kind of message
...maybe something similar needs to be done.
https://github.com/yt-dlp/yt-dlp
There's no option for cookies in my rss reader.
I think the issue is google blacklisting my ISP or IP addresses I'm getting issued.
It's possible. But my point is that specific message about "your computer may be sending automated queries" can be related to that.
Half of Google's "Are you a person" detection is looking for cookies and other web browser signs.
I mean if their captcha page sets a cookie, that might explain why my keyword and bang searches give me a captcha page intermittently but the second attempt always lets me through fine.
But it doesn't explain why neither rss reader nor browser can get any response from an rss feed.
The only common thing there is the network. And the rss feeder was perfectly fine yesterday.
I've come to despise google and resent needing to use their services.
I bet that's the problem. I've always had a lot more captcha demands from Google when I tried to disable their tracking.
I just tried loading a Youtube RSS feed into Akregator and it works, but I've never used Youtube RSS feeds before...I'm sure it's only a matter of time before my IP gets blocked from pulling RSS feeds without a cookie.
Edit: This reddit thread may help you
There is a way to force your ISP to issue you a new IP address that typically works. You'll have to log into your firewall device, and somewhere on the interface/network card properties for the external/internet network interface there may be an option to change the MAC address of that interface. Just change the last digit of the MAC address to something new between 1 and 9 and save the changes, then reboot the firewall or release/renew the external IP address assigned by DHCP. Your ISP should assign a new IP to go with that new MAC address. You can do this multiple times.
That should get you around any blacklistings, though there's no guarantee that their artificial stupids won't auto-flag your new IP address just like they did the old one.
Also beware: Some ISPs lock your account to the MAC of your modem/router and your internet access could break.
When I first signed up with the ISP they gave account details for making direct a ppp connection via the router. They don't allow you to connect directly to the router though and maintain it remotely.
I still use the same method, although I had a little trouble when we moved house as they changed the account details. Most people use the internal DHCP service on the router but I'm glad I'm the sore thumb in this case because I have some control.
Stopping and restarting that connection generally result in a newly leased IP address. It also gets renewed automatically but that usually results in the same IP address. I've never bothered to find out what the length of the lease.