I’m ambivalent about the message here. I agree with the broad strokes: security, scaling, and ensuring reliability are a lot harder than building the initial MVP, and then organisational overheads...
I’m ambivalent about the message here. I agree with the broad strokes: security, scaling, and ensuring reliability are a lot harder than building the initial MVP, and then organisational overheads are far harder to optimise than even the toughest tech problem. But the choice of Google as an example strikes me as an odd one…
I’m not going to go as far as to call it a strawman - I absolutely believe that there are some people out there on the internet claiming that they could build a better search engine with a team of three in a month - but it’s a product that I and any dev I know would immediately classify as “huge, difficult, and uncertain” if anyone even suggested trying it.
On the other hand, I’ve totally been the “I could do that in a weekend” guy on occasion, and I don’t think that’s necessarily wrong either. There’s concrete evidence that the tech for functional equivalents to platforms like Reddit or Twitter can be built with a tiny fraction of the resources in a way that I don’t think search could, for example.
That doesn’t build Reddit or Twitter per se, the value of those platforms is in the community more than the code, but I think that’s part of what prompts devs to say this in the first place: frustration that for a not-overly-difficult codebase, we’re stuck using a broken/expensive/politically dangerous version when we could have something so much better.
For me, it’s a cry of frustration: sometimes it’s because a fault with a piece of niche software is so egregiously bad that I can’t believe it happened in the first place, sometimes it’s because the government spent £35m of our tax money on an app that could not in any possible way justify even 5% of that budget, and sometimes it’s just because I’ve hit the same frustrating bug in a commercial platform three times per day for the past year.
I say it because I am absolutely willing to put my money where my mouth is: pay me even a fraction of what that contract cost and I will build an app that actually works and then sail off on a raft made of banknotes. Open source your platform code, and I will fix that bug that’s been pissing us all off this whole time.
There are enough examples of these things happening, and happening even in wildly more technically or socially complex verticals like OS design or encyclopedia moderation, that I don’t think it’s idle talk a lot of the time. It’s difficult seeing things that could objectively be improved, and knowing they won’t be thanks to inertia, mismatched commercial incentives, or both.
Is there? I think this is not true (1) writing/maintaining software becomes a lot harder at scale (for example CPU bugs which are rare will start to become everyday occurences when your software...
On the other hand, I’ve totally been the “I could do that in a weekend” guy on occasion, and I don’t think that’s necessarily wrong either. There’s concrete evidence that the tech for functional equivalents to platforms like Reddit or Twitter can be built with a tiny fraction of the resources in a way that I don’t think search could, for example.
Is there? I think this is not true
(1) writing/maintaining software becomes a lot harder at scale (for example CPU bugs which are rare will start to become everyday occurences when your software runs on 1,000 servers instead of 1 server, and your software has to handle them)
(2) there's a kind of iceberg effect when it comes to writing software – there's lots of stuff that is necessary and quite complex but end-users don't immediately notice (think moderation, advertising, etc).
(3) at some point your organisation will become larger and this introduces lots of additional complexities and functions (dealing with the law, handling internal HR stuff, etc)
I guess it depends on the lens you look at it through; I was thinking broadly about Tildes/Mastodon when I said that, and meaning that a "functional equivalent" just needs to do roughly the same...
I guess it depends on the lens you look at it through; I was thinking broadly about Tildes/Mastodon when I said that, and meaning that a "functional equivalent" just needs to do roughly the same job for the user rather than being a 1:1 match for the platform as a whole.
I prefer Tildes to Reddit, for example, and I know a lot of people here feel similarly; you're absolutely right that massive scale adds complexity and overhead, but to an extent what I'm questioning is whether scale is actually desirable or beneficial to the users beyond a certain threshold. Maybe some people would find Tildes too small, but there's a whole lot of headroom above where we are right now before technical or organisational scaling issues really start to bite.
I could also point at Hacker News as a concrete example of large scale Reddit-like platform done on a tiny fraction of the resources, but honestly that's not the core of the point I was making. I said right at the start that I agree about a lot of the difficulties the post points out, scaling included, and I don't necessarily think "Hacker News managed it, so you should too" is particularly helpful. I'm more interested in the relative ease of making smaller-scale alternatives as an avenue for increasing user choice than I am in questioning Reddit's specific hiring choices, basically.
Tildes is great, but it's not a functional equivalent to reddit. It's much, much different. At first glance it's similar because the interface looks similar, but the scales are so wildly off that...
Tildes is great, but it's not a functional equivalent to reddit. It's much, much different.
At first glance it's similar because the interface looks similar, but the scales are so wildly off that both sites have totally different problem sets.
I can't use tildes to search for an answer for my specific problem in a niche community dating back over a decade. I can't ask questions about my city in my city's subreddit on tildes, and so on.
The scale alone makes both sites extremely different, not only from an engineering standpoint, but from a functionality standpoint too.
No matter how great tildes is as a product, it cannot directly compete with reddit without the scale reddit has, and thus as an end user, I will continue to use reddit.
The scale has a lot of bad implications, but it has good implications too. No site could ever hope to be a general repository of niche, expert knowledge in as many areas as reddit has without the scale reddit has. Before reddit existed, if I had a question on, say, fish tanks, I'd have to search around and figure out what the good fish tank forums were, then register for an account, then ask my question and probably get flames because I know nothing about that specific fish tank forums posting guidelines, how the site works, or even how to search many times.
On reddit, I just search "fish tanks", find that /r/aquariums is the most popular subreddit by far, and then search or post in that specific subreddit.
No site could ever hope to replicate a similar experience without that scale, and as many flaws as reddit has, it's still pretty valuable as an end user.
That makes sense (although I think it's a different argument), and I think it's compatible with the thesis in the original article: building a small functional prototype is relatively...
That makes sense (although I think it's a different argument), and I think it's compatible with the thesis in the original article: building a small functional prototype is relatively straightforward, building a fully-fledged company which operates at the same scale is much harder.
I could also point at Hacker News as a concrete example of large scale Reddit-like platform done on a tiny fraction of the resources
I think Hacker News doesn't run at the same scale – it has something like 5mn monthly active users whereas Reddit has 73.1 million daily active unique users according to their SEC filings. Reddit also has a more complex data model (e.g. subreddits, lots of images, a whole notification system which is basically a big distributed system mess) and does ads (whereas Hacker News doesn't need to make money because it's funded by YCombinator).
I don't agree; both Twitter and Reddit are very hard sites to actually run at scale. You can get a prototype up, but it's not one that will stand 1M MAUs. Twitter in particular has a brutal job -...
I don't agree; both Twitter and Reddit are very hard sites to actually run at scale. You can get a prototype up, but it's not one that will stand 1M MAUs.
Twitter in particular has a brutal job - you have vast dichotomy of data behavior that you need to support, the broadcast kind, where a big shot (e.g Taylor Swift, POTUS, whatever) tweets and you have to get that tweet into a billion user's inbox at the same time, and the opposite, the kind of users that tweet way more than they read.
There's a reason Twitter is the first case study in the Designing Data Intensive Software book.
Reddit has an easier job, because subreddit is a natural sharding spot, but even then the moment you add multimedia the scaling becomes quite brutal. Just look at the now dead cohost - sounds simple, but they were burning 40k a month on server costs for just 20k MAU.
Making software that works at scale is hard, especially if you want to support things that are tablestakes to many users like multimedia. Tildes made a smart decision by being text-only, but it's not a decision that really works for all kinds of sites.
I'm pretty much with you - I wasn't trying to say scaling is easy, more that web search is inherently hard regardless of user volume, whereas social platforms have a much lower technical barrier...
I'm pretty much with you - I wasn't trying to say scaling is easy, more that web search is inherently hard regardless of user volume, whereas social platforms have a much lower technical barrier to entry. I also commented at about the same time as you with a bit more on why I wasn't necessarily considering massive scale as part of what makes up a valuable equivalent or replacement for the user.
Watching BlueSky scale up is the closest we’re likely to get to finding out what happens if someone else tries to rebuild Twitter. I think they’re doing a good job of showing that it’s possible to...
Watching BlueSky scale up is the closest we’re likely to get to finding out what happens if someone else tries to rebuild Twitter.
I think they’re doing a good job of showing that it’s possible to do it better and more efficiently than Twitter did (having learned a few things), but also, they’ve been working on it for quite a while and they still have a lot of challenges ahead of them. Definitely not a weekend project.
That’s quite far off, there are less than a billion domains. And kind of obviously so, the world isn’t spending several trillion dollars on domain name registries. At $10 per domain, domain name...
I'd guess that we can find on the order a trillion domains.
That’s quite far off, there are less than a billion domains. And kind of obviously so, the world isn’t spending several trillion dollars on domain name registries. At $10 per domain, domain name registration would be equivalent to more than the GDP of Germany, the 3rd largest economy on the planet, and around half of China’s GDP.
This is a bit of a sidebar, but I have a hard time taking people seriously when they write about tech and have terrible readability on their website. body { max-width:50rem; margin:1rem auto;...
This is a bit of a sidebar, but I have a hard time taking people seriously when they write about tech and have terrible readability on their website.
body { max-width:50rem; margin:1rem auto; line-height:1.3; }
Suddenly the site looks approximately the same, but wonderful on just about every device.
Edit: typically I wouldn't really care or bring this up, but I think the message from this author is actually an important one - regular people don't really understand what it takes to make most software projects work. They don't understand what is hard, what is easy. And I think the process of moving from junior dev (or hobby dev) to a senior dev is the process of sussing all of that out.
Yes, it’s not very readable, but Dan Luu has been blogging since 2013 and I’m sure he’s gotten criticism about his website’s design before. I think at this point, it has to be seen as a deliberate...
Yes, it’s not very readable, but Dan Luu has been blogging since 2013 and I’m sure he’s gotten criticism about his website’s design before. I think at this point, it has to be seen as a deliberate choice.
I definitely believe he is making a choice, it's just a shame that he is making an obnoxious usability choice. While I don't recall anything he has written that is directly related to usability...
I definitely believe he is making a choice, it's just a shame that he is making an obnoxious usability choice. While I don't recall anything he has written that is directly related to usability for end users like this, I think it detracts from the things that he writes about.
I think the design itself is fine, maybe even great - it's so simple, and is mostly preserved by the CSS rule I proposed, which won't really change much substantive about the site, but make it readable on large screens.
I’m ambivalent about the message here. I agree with the broad strokes: security, scaling, and ensuring reliability are a lot harder than building the initial MVP, and then organisational overheads are far harder to optimise than even the toughest tech problem. But the choice of Google as an example strikes me as an odd one…
I’m not going to go as far as to call it a strawman - I absolutely believe that there are some people out there on the internet claiming that they could build a better search engine with a team of three in a month - but it’s a product that I and any dev I know would immediately classify as “huge, difficult, and uncertain” if anyone even suggested trying it.
On the other hand, I’ve totally been the “I could do that in a weekend” guy on occasion, and I don’t think that’s necessarily wrong either. There’s concrete evidence that the tech for functional equivalents to platforms like Reddit or Twitter can be built with a tiny fraction of the resources in a way that I don’t think search could, for example.
That doesn’t build Reddit or Twitter per se, the value of those platforms is in the community more than the code, but I think that’s part of what prompts devs to say this in the first place: frustration that for a not-overly-difficult codebase, we’re stuck using a broken/expensive/politically dangerous version when we could have something so much better.
For me, it’s a cry of frustration: sometimes it’s because a fault with a piece of niche software is so egregiously bad that I can’t believe it happened in the first place, sometimes it’s because the government spent £35m of our tax money on an app that could not in any possible way justify even 5% of that budget, and sometimes it’s just because I’ve hit the same frustrating bug in a commercial platform three times per day for the past year.
I say it because I am absolutely willing to put my money where my mouth is: pay me even a fraction of what that contract cost and I will build an app that actually works and then sail off on a raft made of banknotes. Open source your platform code, and I will fix that bug that’s been pissing us all off this whole time.
There are enough examples of these things happening, and happening even in wildly more technically or socially complex verticals like OS design or encyclopedia moderation, that I don’t think it’s idle talk a lot of the time. It’s difficult seeing things that could objectively be improved, and knowing they won’t be thanks to inertia, mismatched commercial incentives, or both.
Is there? I think this is not true
I guess it depends on the lens you look at it through; I was thinking broadly about Tildes/Mastodon when I said that, and meaning that a "functional equivalent" just needs to do roughly the same job for the user rather than being a 1:1 match for the platform as a whole.
I prefer Tildes to Reddit, for example, and I know a lot of people here feel similarly; you're absolutely right that massive scale adds complexity and overhead, but to an extent what I'm questioning is whether scale is actually desirable or beneficial to the users beyond a certain threshold. Maybe some people would find Tildes too small, but there's a whole lot of headroom above where we are right now before technical or organisational scaling issues really start to bite.
I could also point at Hacker News as a concrete example of large scale Reddit-like platform done on a tiny fraction of the resources, but honestly that's not the core of the point I was making. I said right at the start that I agree about a lot of the difficulties the post points out, scaling included, and I don't necessarily think "Hacker News managed it, so you should too" is particularly helpful. I'm more interested in the relative ease of making smaller-scale alternatives as an avenue for increasing user choice than I am in questioning Reddit's specific hiring choices, basically.
[Edit] Cleanup and clarifications.
Tildes is great, but it's not a functional equivalent to reddit. It's much, much different.
At first glance it's similar because the interface looks similar, but the scales are so wildly off that both sites have totally different problem sets.
I can't use tildes to search for an answer for my specific problem in a niche community dating back over a decade. I can't ask questions about my city in my city's subreddit on tildes, and so on.
The scale alone makes both sites extremely different, not only from an engineering standpoint, but from a functionality standpoint too.
No matter how great tildes is as a product, it cannot directly compete with reddit without the scale reddit has, and thus as an end user, I will continue to use reddit.
The scale has a lot of bad implications, but it has good implications too. No site could ever hope to be a general repository of niche, expert knowledge in as many areas as reddit has without the scale reddit has. Before reddit existed, if I had a question on, say, fish tanks, I'd have to search around and figure out what the good fish tank forums were, then register for an account, then ask my question and probably get flames because I know nothing about that specific fish tank forums posting guidelines, how the site works, or even how to search many times.
On reddit, I just search "fish tanks", find that /r/aquariums is the most popular subreddit by far, and then search or post in that specific subreddit.
No site could ever hope to replicate a similar experience without that scale, and as many flaws as reddit has, it's still pretty valuable as an end user.
That makes sense (although I think it's a different argument), and I think it's compatible with the thesis in the original article: building a small functional prototype is relatively straightforward, building a fully-fledged company which operates at the same scale is much harder.
I think Hacker News doesn't run at the same scale – it has something like 5mn monthly active users whereas Reddit has 73.1 million daily active unique users according to their SEC filings. Reddit also has a more complex data model (e.g. subreddits, lots of images, a whole notification system which is basically a big distributed system mess) and does ads (whereas Hacker News doesn't need to make money because it's funded by YCombinator).
I don't agree; both Twitter and Reddit are very hard sites to actually run at scale. You can get a prototype up, but it's not one that will stand 1M MAUs.
Twitter in particular has a brutal job - you have vast dichotomy of data behavior that you need to support, the broadcast kind, where a big shot (e.g Taylor Swift, POTUS, whatever) tweets and you have to get that tweet into a billion user's inbox at the same time, and the opposite, the kind of users that tweet way more than they read.
There's a reason Twitter is the first case study in the Designing Data Intensive Software book.
Reddit has an easier job, because subreddit is a natural sharding spot, but even then the moment you add multimedia the scaling becomes quite brutal. Just look at the now dead cohost - sounds simple, but they were burning 40k a month on server costs for just 20k MAU.
Making software that works at scale is hard, especially if you want to support things that are tablestakes to many users like multimedia. Tildes made a smart decision by being text-only, but it's not a decision that really works for all kinds of sites.
I'm pretty much with you - I wasn't trying to say scaling is easy, more that web search is inherently hard regardless of user volume, whereas social platforms have a much lower technical barrier to entry. I also commented at about the same time as you with a bit more on why I wasn't necessarily considering massive scale as part of what makes up a valuable equivalent or replacement for the user.
This really resonates with me. I tend to look for open source versions of things precisely for this reason.
Watching BlueSky scale up is the closest we’re likely to get to finding out what happens if someone else tries to rebuild Twitter.
I think they’re doing a good job of showing that it’s possible to do it better and more efficiently than Twitter did (having learned a few things), but also, they’ve been working on it for quite a while and they still have a lot of challenges ahead of them. Definitely not a weekend project.
That’s quite far off, there are less than a billion domains. And kind of obviously so, the world isn’t spending several trillion dollars on domain name registries. At $10 per domain, domain name registration would be equivalent to more than the GDP of Germany, the 3rd largest economy on the planet, and around half of China’s GDP.
https://investor.verisign.com/news-releases/news-release-details/verisign-reports-internet-has-3540-million-domain-name
This is a bit of a sidebar, but I have a hard time taking people seriously when they write about tech and have terrible readability on their website.
body { max-width:50rem; margin:1rem auto; line-height:1.3; }Suddenly the site looks approximately the same, but wonderful on just about every device.
Edit: typically I wouldn't really care or bring this up, but I think the message from this author is actually an important one - regular people don't really understand what it takes to make most software projects work. They don't understand what is hard, what is easy. And I think the process of moving from junior dev (or hobby dev) to a senior dev is the process of sussing all of that out.
Yes, it’s not very readable, but Dan Luu has been blogging since 2013 and I’m sure he’s gotten criticism about his website’s design before. I think at this point, it has to be seen as a deliberate choice.
I definitely believe he is making a choice, it's just a shame that he is making an obnoxious usability choice. While I don't recall anything he has written that is directly related to usability for end users like this, I think it detracts from the things that he writes about.
I think the design itself is fine, maybe even great - it's so simple, and is mostly preserved by the CSS rule I proposed, which won't really change much substantive about the site, but make it readable on large screens.