I feel like someone here asks in a semi monthly basis about how RSS works or if it’s worth it to use it. This is a good starting resource on how it works, how to get started, and why you’d want to.
I feel like someone here asks in a semi monthly basis about how RSS works or if it’s worth it to use it.
This is a good starting resource on how it works, how to get started, and why you’d want to.
IMO the prevalence of LLM scrapers will kill it within a few years. RSS is perfect way to scrape all of the information on a site, and it’s very difficult to block scraping bots while not blocking...
IMO the prevalence of LLM scrapers will kill it within a few years. RSS is perfect way to scrape all of the information on a site, and it’s very difficult to block scraping bots while not blocking RSS readers, which would kinda obviate the point.
RSS has some pretty decent options for caching too eTags etc Rachel even has a project measuring readers that are (mis)behaving: https://rachelbythebay.com/frb/ and the site blocks the bad ones...
RSS has some pretty decent options for caching too eTags etc
Rachel even has a project measuring readers that are (mis)behaving: https://rachelbythebay.com/frb/ and the site blocks the bad ones pretty quickly.
There's no need to fetch the RSS feed every 60 seconds when it changes like once a week. You use last-modified and an etag and manage the response properly.
This is great! I'm sometimes working on my own reader, and things like this is something I'm worried about. I kinda wish the list was prioritized, but this will be really helpful in its current...
This is great! I'm sometimes working on my own reader, and things like this is something I'm worried about. I kinda wish the list was prioritized, but this will be really helpful in its current form as well
I moved over to using an RSS reader when Reddit started doing its thing with the apps and I can't imagine going back now. You just have so much more control over your internet experience. And I...
I moved over to using an RSS reader when Reddit started doing its thing with the apps and I can't imagine going back now. You just have so much more control over your internet experience. And I think getting away from the algorithms is such a big part of it. I just see the things that I want to see, and it's all in chronological order. That and the general moving away from reading big content firehoses is a great improvement. Blogs are great and they never went away!
Pro tip: you can get a private feed of your Reddit front page from the settings. I grab that and do some enrichment magic on it to get all the videos/images/text in it before feeding it to FreshRSS.
Pro tip: you can get a private feed of your Reddit front page from the settings.
I grab that and do some enrichment magic on it to get all the videos/images/text in it before feeding it to FreshRSS.
Good tip. I ended up just paring down to the subreddits I actually cared about and adding their individual RSS feeds. But didn't know you could also get one for your whole front page. Also 👍 to...
Good tip. I ended up just paring down to the subreddits I actually cared about and adding their individual RSS feeds. But didn't know you could also get one for your whole front page. Also 👍 to FreshRSS. Has become one of my most-used services.
A tip of my own. If anyone is wanting to add individual subreddits as feeds, they also support getting the top posts of a period as you can in the reddit interface. For example: https://www.reddit.com/r/popheads/top.rss?t=week
Great for subreddits I care about, but I don't actually want to see every single post, just the best stuff of whatever time period I want to check it.
Ooh! That's perfect. Especially all LLM-themed subs are complete cesspools if you follow them "live", getting some delay on them might be a good solution. I had to pare down my front page really...
Ooh! That's perfect. Especially all LLM-themed subs are complete cesspools if you follow them "live", getting some delay on them might be a good solution.
I had to pare down my front page really heavily just because I had some low-quality ones in there that were really easy just to pass over when reading on my browser, but when they were front and center they got annoying.
Could you please share the enrichment magic that you perform? Also would you recommend FreshRSS over other RSS clients? Does it behave well on mobile browsers?
Could you please share the enrichment magic that you perform?
Also would you recommend FreshRSS over other RSS clients? Does it behave well on mobile browsers?
I use NetNewsWire as a client to FreshRSS so I get a native experience, it has direct support for FreshRSS (and others). Basically they emulate the Ye Olde Google Reader API. Never used the web ui...
I use NetNewsWire as a client to FreshRSS so I get a native experience, it has direct support for FreshRSS (and others). Basically they emulate the Ye Olde Google Reader API. Never used the web ui on mobile browser.
I did do some research and ended up with FreshRSS because it's in active development, at the time I made the jump the closest competitor Tiny Tiny RSS was in a development limbo.
Both support plugins, but what finally won FreshRSS for me was the ability to generate feeds from HTML using XPath or CSS selectors: https://danq.me/2022/09/27/freshrss-xpath/
My current enrichment magic is this project: https://github.com/lepinkainen/feed-forge (It's a vibe coding experiment, works pretty well for me, but there's still definite jank that wouldn't be there if it was artisanally coded =P )
My previous projects for Hacker News and Reddit started to become so similar I combined them to the feed forge project.
For Hacker news it fetches the opengraph preview for the site + tries to keep the comment and vote count on the item updated in the feed for a while. Just looking at the HN titles is kinda useless when trying to decide whether it's worth opening. With a preview + vote + comment count I can make that decision.
For Reddit I first tried to use their API to fetch my personal front page, but it was a complete mess (the code is still there), then I (re-)discovered the custom private front page feed and wrote a parser for that. There's a bunch of special handling for videos, galleries etc trying to open the comments url for each + generate a somewhat intelligent preview for them.
Lobster.rs and Tildes are coming up next, but I'm using my Claude Code quota for other projects so those are on hold =)
All of this is perfectly doable with something like a self-hosted n8n instance - might actually be easier - but I'm a backend coder by heart so drawing boxes feels wrong when I can write the code.
Has FreshRSS ever fixed the issue that you can't sort anything by published date and not Date added? I tried FreshRSS a like around 2019 or 2020 and I wasn't happy with it because of the lack of...
Has FreshRSS ever fixed the issue that you can't sort anything by published date and not Date added?
I tried FreshRSS a like around 2019 or 2020 and I wasn't happy with it because of the lack of ability to sort by published date and not date added. I subscribed to this issue and at the time was further put off by the developer being dismissive of many user's valid requests for this use case and the vast amount of people replying to this issue, emoting to it, and the vast amount of other topics being merged into it also asking for this feature, spanning all the way to 2019.
I remember browsing the code and wondering what could possibly make this feature so difficult since sorting ability is one of the most basic things of most aggregator type software. And the software pulls the published date from somewhere, it's just tying that published date variable to a sort order. I ended up moving on way before I came to any solutions.
I just re-read the thread and apparently he did develop the feature earlier this year, so I'll try it again when I get home.
It seems like all of the major self-hosted RSS readers are based on some 20 year old php codebase - which brings its own issues :D ...I don't need to write my own. I don't need to write my own. I...
It seems like all of the major self-hosted RSS readers are based on some 20 year old php codebase - which brings its own issues :D
...I don't need to write my own. I don't need to write my own. I don't need to write my own...
It never came up on my searches, but I'll check it out. It being a Go application is a massive plus for me :) And it seems it has special handling for some sites too. EDIT: Ooh and Postgresql for...
It never came up on my searches, but I'll check it out. It being a Go application is a massive plus for me :)
And it seems it has special handling for some sites too.
EDIT: Ooh and Postgresql for a backend. I might be in love =)
Well since NewsNetWire came up I thought I'd ask... I'm an RSS novice, and I'm wondering if there is a way to add an individual feed to multiple folders, or to tag? It seems like this is possible...
Well since NewsNetWire came up I thought I'd ask...
I'm an RSS novice, and I'm wondering if there is a way to add an individual feed to multiple folders, or to tag? It seems like this is possible from reading online but I can't figure out how.
I don't think many systems support it. Newsblur does though, I still pay for it because it's the one I moved to when Google Reader died and I want to support it :) ...and now that I logged in, it...
I don't think many systems support it.
Newsblur does though, I still pay for it because it's the one I moved to when Google Reader died and I want to support it :)
...and now that I logged in, it seems there are some rarely updating feeds I haven't transferred to FreshRSS yet =)
I'm so happy that RSS is getting some love. It's such an amazing protocol that has you go beyond the walled gardens, or needing to create an account just to follow someone(s). I remember when...
I'm so happy that RSS is getting some love. It's such an amazing protocol that has you go beyond the walled gardens, or needing to create an account just to follow someone(s).
I remember when Twitter used to have RSS; and for the longest time I didn't need an account. So of course they quietly killed that.
But yes, highly recommend looking into RSS.
...Yes, still mad about the demise of Google Reader.
My usage ended when they killed even opengraph previews of tweets at the same time they killed the API "to prevent bots" Which, in fact, didn't prevent any bots, just a bunch of cool and silly...
My usage ended when they killed even opengraph previews of tweets at the same time they killed the API "to prevent bots"
Which, in fact, didn't prevent any bots, just a bunch of cool and silly bots that posted automated content like the price of beer at a bar or random quotes. And my bot that translated Twitter URLs on IRC to an actual preview.
Thank you for the information! I was aware of RSS before, but never had a strong need to implement it. I just setup FreshRSS and added a few subreddits and podcasts.
Thank you for the information! I was aware of RSS before, but never had a strong need to implement it. I just setup FreshRSS and added a few subreddits and podcasts.
I feel like someone here asks in a semi monthly basis about how RSS works or if it’s worth it to use it.
This is a good starting resource on how it works, how to get started, and why you’d want to.
My biggest concern with RSS is that if it catches on too much everyone will kill it because they can't control it as easily.
IMO the prevalence of LLM scrapers will kill it within a few years. RSS is perfect way to scrape all of the information on a site, and it’s very difficult to block scraping bots while not blocking RSS readers, which would kinda obviate the point.
I think if the LLMs opted to scrape the RSS feed instead of your actual webpage most site owners would be happy for the reduced hosting bill.
RSS has some pretty decent options for caching too eTags etc
Rachel even has a project measuring readers that are (mis)behaving: https://rachelbythebay.com/frb/ and the site blocks the bad ones pretty quickly.
There's no need to fetch the RSS feed every 60 seconds when it changes like once a week. You use last-modified and an etag and manage the response properly.
This is great! I'm sometimes working on my own reader, and things like this is something I'm worried about. I kinda wish the list was prioritized, but this will be really helpful in its current form as well
I moved over to using an RSS reader when Reddit started doing its thing with the apps and I can't imagine going back now. You just have so much more control over your internet experience. And I think getting away from the algorithms is such a big part of it. I just see the things that I want to see, and it's all in chronological order. That and the general moving away from reading big content firehoses is a great improvement. Blogs are great and they never went away!
Pro tip: you can get a private feed of your Reddit front page from the settings.
I grab that and do some enrichment magic on it to get all the videos/images/text in it before feeding it to FreshRSS.
Good tip. I ended up just paring down to the subreddits I actually cared about and adding their individual RSS feeds. But didn't know you could also get one for your whole front page. Also 👍 to FreshRSS. Has become one of my most-used services.
A tip of my own. If anyone is wanting to add individual subreddits as feeds, they also support getting the top posts of a period as you can in the reddit interface. For example: https://www.reddit.com/r/popheads/top.rss?t=week
Great for subreddits I care about, but I don't actually want to see every single post, just the best stuff of whatever time period I want to check it.
Ooh! That's perfect. Especially all LLM-themed subs are complete cesspools if you follow them "live", getting some delay on them might be a good solution.
I had to pare down my front page really heavily just because I had some low-quality ones in there that were really easy just to pass over when reading on my browser, but when they were front and center they got annoying.
Could you please share the enrichment magic that you perform?
Also would you recommend FreshRSS over other RSS clients? Does it behave well on mobile browsers?
I use NetNewsWire as a client to FreshRSS so I get a native experience, it has direct support for FreshRSS (and others). Basically they emulate the Ye Olde Google Reader API. Never used the web ui on mobile browser.
I did do some research and ended up with FreshRSS because it's in active development, at the time I made the jump the closest competitor Tiny Tiny RSS was in a development limbo.
Both support plugins, but what finally won FreshRSS for me was the ability to generate feeds from HTML using XPath or CSS selectors: https://danq.me/2022/09/27/freshrss-xpath/
My current enrichment magic is this project: https://github.com/lepinkainen/feed-forge (It's a vibe coding experiment, works pretty well for me, but there's still definite jank that wouldn't be there if it was artisanally coded =P )
My previous projects for Hacker News and Reddit started to become so similar I combined them to the feed forge project.
For Hacker news it fetches the opengraph preview for the site + tries to keep the comment and vote count on the item updated in the feed for a while. Just looking at the HN titles is kinda useless when trying to decide whether it's worth opening. With a preview + vote + comment count I can make that decision.
For Reddit I first tried to use their API to fetch my personal front page, but it was a complete mess (the code is still there), then I (re-)discovered the custom private front page feed and wrote a parser for that. There's a bunch of special handling for videos, galleries etc trying to open the comments url for each + generate a somewhat intelligent preview for them.
Lobster.rs and Tildes are coming up next, but I'm using my Claude Code quota for other projects so those are on hold =)
All of this is perfectly doable with something like a self-hosted n8n instance - might actually be easier - but I'm a backend coder by heart so drawing boxes feels wrong when I can write the code.
Has FreshRSS ever fixed the issue that you can't sort anything by published date and not Date added?
I tried FreshRSS a like around 2019 or 2020 and I wasn't happy with it because of the lack of ability to sort by published date and not date added. I subscribed to this issue and at the time was further put off by the developer being dismissive of many user's valid requests for this use case and the vast amount of people replying to this issue, emoting to it, and the vast amount of other topics being merged into it also asking for this feature, spanning all the way to 2019.
I remember browsing the code and wondering what could possibly make this feature so difficult since sorting ability is one of the most basic things of most aggregator type software. And the software pulls the published date from somewhere, it's just tying that published date variable to a sort order. I ended up moving on way before I came to any solutions.
I just re-read the thread and apparently he did develop the feature earlier this year, so I'll try it again when I get home.
It seems like all of the major self-hosted RSS readers are based on some 20 year old php codebase - which brings its own issues :D
...I don't need to write my own. I don't need to write my own. I don't need to write my own...
Damnit.
Have you tried miniflux? I like it in part because it is not based on some 20 year old php codebase. I've been using it for years and I quite like it
It never came up on my searches, but I'll check it out. It being a Go application is a massive plus for me :)
And it seems it has special handling for some sites too.
EDIT: Ooh and Postgresql for a backend. I might be in love =)
Right? haha Here I am thinking the same thing myself!
Well since NewsNetWire came up I thought I'd ask...
I'm an RSS novice, and I'm wondering if there is a way to add an individual feed to multiple folders, or to tag? It seems like this is possible from reading online but I can't figure out how.
I don't think many systems support it.
Newsblur does though, I still pay for it because it's the one I moved to when Google Reader died and I want to support it :)
...and now that I logged in, it seems there are some rarely updating feeds I haven't transferred to FreshRSS yet =)
I'm so happy that RSS is getting some love. It's such an amazing protocol that has you go beyond the walled gardens, or needing to create an account just to follow someone(s).
I remember when Twitter used to have RSS; and for the longest time I didn't need an account. So of course they quietly killed that.
But yes, highly recommend looking into RSS.
...Yes, still mad about the demise of Google Reader.
My usage ended when they killed even opengraph previews of tweets at the same time they killed the API "to prevent bots"
Which, in fact, didn't prevent any bots, just a bunch of cool and silly bots that posted automated content like the price of beer at a bar or random quotes. And my bot that translated Twitter URLs on IRC to an actual preview.
Thank you for the information! I was aware of RSS before, but never had a strong need to implement it. I just setup FreshRSS and added a few subreddits and podcasts.