Sub-tildes have a fundamental problem. Can it be solved?
I've been doing some thinking and concluded that in it's current form, tildes design has a fundamental problem that is going to make high-quality discussions nearly impossible.
Assumptions
I'm relying on four assumptions here:
- A discussion's quality is proportional to it's exclusiveness. In other words, the more wide the audience participating in a discussion is, the worse the discussion gets. It's not hard to see this. A discussion about a discovery in cancer reasearch on a news site will be much lower quality than among cancer reasearchers. This has also been shown to be true by reddit's /r/all.
- tildes get more specific, the "deeper" they are in the hierachy. ~sci.biology.cancer is more specific than ~sci. ~sci also a has more subscribers.
- tildes.net wants to use this specialization to foster high-quality and qualified discussions on specific topics.
- tilde submissions "bubble up", as they currently do
The Problem
You might be able to see what I'm getting at. I think these three together are a fundamental problem for the quality of discussion in subgroups:
- Highly upvoted posts from specific subgroups will be exposed to wider audiences, thus lowering the quality of discussion.
- More generic posts have a higher likelihood of receiving upvotes from the more general groups above them, thus lowering the quality of submissions.
A Scenario
Let's simulate a scenario using my above assumptions. This might be unhelpful, since it's very easy to poke holes in such a specific scenario. This is more intended as an overall picture of the incentives the users have.
We have three submissions to ~sci.biology.cancer, about the news of three different discoveries:
- A link to an original scientific paper with it's original title
- A link to an original scientific paper, with a modified title
- A link to a news story in a popular tabloid newspaper, with it's clickbait title
So, how would these fare?
- The first submission would be upvoted by ~sci.biology.cancer subscribers, who understand the paper and topic, but are low in numbers.
- The second submission would be upvoted by ~sci.biology, who are familiar enough to understand the modified title.
- The third submission can be understood by anyone, and would be upvoted by the whole of ~sci, slingshotting to the top.
Let's take at the result in ~sci.biology.cancer:
The highest ranked post is now a clickbait article of no significant interest to anyone actually knowledgeable about the topic, filled with unqualified discussion. The second ranked post is slightly better, but still less useful than the first post, which is being drowned out by other submissions.
Conclusion
As a submitter with the current system, instead of submitting high quality content that interests the subtilde, it is in your interest to submit a post that will appeal to the lowest common denominator, the subtildes above you. This will significantly decrease the quality of specialized subtildes.
Ideas
I believe the bubbling up mechanic must be modified in some way to prevent this unfortunate systemic issue. I don't really have a good solution, but here's some ideas to get the brainstorming going:
- No participation (voting/commenting) for users higher up the chain. This would be very extreme.
- users don't see comments made higher-up the chain. ~sci.biology would not see ~sci comments. This would be extraordinarily confusing and have weird edge cases.
- Votes would be counted separately for each part of the sub-tilde chain. A post might be highly upvoted in ~sci, but only receive a few upvotes in ~sci.biology. I like this idea in general, but it does not solve the problem of the low-quality responses landing in ~sci.biology.cancer too. Maybe that's just an acceptable trade-off, though.
What are your thoughts on this?
The thing you missed is that ~sci and ~sci.biology would not see posts from ~sci.biology.cancer initially. The posts have to bubble up first by being well liked in ~sci.biology.cancer. Posts that are not like by ~sci.biology.cancer will not be seen outside of it. Also, by the time a post bubbles up, it will have had several quality top level comments witch will hopefully discourage lower quality top level comments, or at the very least hide them.
I didn't miss out on that, it's actually part of the problem. Posts that are well-liked will be victims of their own success, discussion wise. The existence of high-quality comments discouraging low-quality posts might reduce this problem though, agreed.
I am also reluctant toward the bubbling system, but I think the speed of post growths can be balanced somewhat to provide an incubating effect where high-quality, specific discussion can generally avoid being drowned out.
It is, at least initially, worth trying. I appreciate that once infrastructure is in place the odds get much lower that it will be revised away from explosive growth dynamics, but even so--
That just postpones the problem. The post would still receive a lot of low-quality attention once it bubbles up.
IMO your 'simulation' is assuming a great deal about the bubble up mechanic (when it's not even fully fleshed out yet) and you're failing to account for some other key mechanics that would counteract the very things you're worried about; namely the trust system and the vote weight and trusted user actions associated with it.
With vote weight, the votes from users of the higher level groups would count for significantly less (perhaps even a minuscule amount) than the trusted users from the group where the article was submitted, e.g. ~sci.biology.cancer. This weight counts for the comments section as well. And there has also been talk of giving trusted members of groups "exemplary votes" with even more added weight (or maybe even a vote weight modifier applied to all other votes on said comment), which they get a set amount of per day/week, and can only use within the group they have acquired their trust. So if things start going off the rails the ~sci.biology.cancer users can simply exemplary vote comments they feel are more worthy than the horde of perhaps less informed users.
And also with the trust system come "mod" like actions for trusted users. With trust, ideally a large % of the ~sci.biology.cancer users will have access to "mod" like actions to help keep the comments on that post in check as well. Those actions will likely also include thing like title editing, link editing, tag editing, etc. so your whole worry about "clickbait" is rendered a bit moot.
I think vote weight is a great idea to counteract this, since it puts submissions to deeper groups at a big disadvantage. Thus, it would make sense to put your post in the least specific group where you think it will get upvotes. So the tabloid poster would have the highest chance of success if they posted in ~sci instead of the more specific group.
My scenario is based on how they currently work and how I've heard @Deimos describe the plan for them. AFAICT, nobody has mentioned this issue yet, so that's why I decided to write a post on it.
It's very likely submitting a cancer story, even directly to ~sci will result it in automatically being moved to ~sci.biology.cancer where it is most relevant. That's one of the potentially powerful things about the tagging and trusted user actions system. Even if the poster submits the article to ~sci, the trusted users there will tag it appropriately and if the subject of the article is clearly cancer, then it will be tagged as such and likely moved to ~sci.biology.cancer as a result. From there the more "expert" trusted users of ~sci.biology.cancer can do what they feel they need to so it lives up to their standards, either by changing the title to be less clickbait or perhaps even changing the link entirely (or using it as the basis for a megathread on the subject) to be better informed on the subject. There is ideally going to be a filter-down and then back up approach to all this.
But again, all the systems are merely theoretical at this point so please don't take my comments as entirely dismissive of your worries. You bringing up the potential pitfalls of the system is a good thing. A great thing, in fact. But I think you're just lacking a bit of understanding regarding some key elements of the system and their potential interactivity, is all.
p.s. And that's not entirely your fault either BTW... I know it's difficult to get a grasp on all this stuff when there is no full documentation written on these systems yet. Discussions on them right now are scattered all over the place and hard to find (especially without search yet), but that will hopefully be changing soon. @Amarok is already working on a more comprehensive FAQ and Mechanics (Future) page as we speak.
Speaking of the devil: https://tildes.net/~tildes/1m1/sub_tildes_have_a_fundamental_problem_can_it_be_solved#comment-fxn :P
Regarding the moving specifically, I think that might be a bit detrimental. An article can be written and discussion can be had at a number of levels of sophistication, and that there is value in that. Lumping these together might be a disservice to both groups. Of course, care would need to be given that duplicates don't dominate.
Yeah it's possible that it will be detrimental but I think so long as the more niche groups are cognizant of the fact that laymen will still need/want to understand the discussion then it won't be too bad.
That's why the idea that was suggested for megathreads is such a good one, IMO. It allows for exactly what you suggest, a chance for understanding the subject from multiple levels of sophistication by providing multiple links that address each level (e.g. include a well written popsci article on the subject + a more serious article on the subject + a peer reviewed journal editorial on the subject + a link to the actual study all of them are based on).
However I do think handing control of the comments section over to the people who know the subject best, for moderation's sake, is still the best approach. That way users with no firm grasp of the subject can still ask questions of the experts but wild speculation from uninformed perspectives can be squashed.
I imagine it working a bit differently than your example. You're underestimating the complexity here and the social group dynamics.
Stuff starts off at ~group.one.two.three.four and it is exclusive to that group. Nobody else gets to see it. Most stuff there (think 99/100 or even 199/200) won't meet the 'quality' threshold to bubble up. In order to level up into group three, it's got to either collect enough exemplary votes or be singled out directly by the curator team in group four as a 'best of' style submission. It might take it days instead of hours to get to that point, but if it's something amazing it might be able to pull it off in an afternoon. That'll depend on the mechanics - and the curators in group three are also going to be a part of this process somehow.
Whatever the mechanism, the post hits the threshold and bubbles up into group three.
Now, something different happens in the comments. The existing good discussion in group four might end up on its own tab, or we might give those threads more clout over new arrivals, or we might do some mixing of the comments from four and three - there are many possibilities, we need to experiment with it and find what works. Either way, though, we know the difference, we know who posted to it in group four vs group three (and so on with two, one, and the top level). That information will help figure out how to thread everything.
Also, in group three, it doesn't have to appear as an old thread. It could appear as a new submission, or the same way it did in four just now visible in three, with fancier comment sorting, or it could start out 'exemplary' in group three already. That's probably going to depend on the users of that hierarchy and how they want this to operate, and what the curator teams do with the post. I imagine different mechanisms for different content.
Now the group three users make their contributions and votes on it. These could be weaker than four, or stronger than four, or independent of four, cumulative with four - again, lots of options available. At this point group three has to decide if it's going to level up, and then so will two, and one, and the root group.
I doubt there will be anything simple at all about this system, and I expect it's going to work differently in each hierarchy. All of those variables are going to be up for grabs in the code, and the curator teams on top of that will be exerting influence of their own.
Also, every level up, the group is older, larger, and yet more heavily moderated as a result. By then those higher groups will be years older, with standards, rules, systems of their own. If you want to get away with submitting fluff, chances are you can only pull that off in smaller, younger groups - further up the chain, the teams will be more fully formed and mature, especially nearer the root - and they'll be operating on a mandate of enforcing quality themselves. It's very likely they'll 'cast down' the submission into a sub-group or put it in some kind of also-ran bit-bucket such as a 'fluff' tab depending on how we set up to handle that.
Honestly what I worry about the most is if a parent spins off a sub group and then fights having that content bubble up the hierarchy. We can't have adversarial relationships like that if this is going to work.
I think that system might provide unfair advantage to more popular subgroups
take for example ~games.traditional.card.mtg vs ~ games.traditional.card.cardsagainsthumanity
given the lack of strategy elements in cah, popular posts there would probably just be funny card combinations.
an exemplary mtg post might be something like "MaRo announces retirement, bids farewell to loyal fans"
so let's say we have an exemplary mtg post trying to bubble up through the games tree vs an exemplary cah post. which do you think will have the advantage? The funny joke posts will likely appeal to a wider audience, so while the mtg post might make it up through .card to .traditional, the cah post probably has a better chance to make it up through to .games
will groups be weighted differently?
will low quality potential groups like .cardsagainsthumanity just not be allowed to exist?
The hierarchy is going to need periodic reorganization as it grows. In the example you provided, it might be necessary to add a .card.strategy.* and put the strategy games under that one. The structure isn't meant to be set in stone, it'll have to evolve as the site does.
There is Godwin's Law, there's Murphy's Law, there's Sturgeon's Law... I think in this pantheon of laws there should be added a law that for whatever you say ever, in an internet forum, it will be met with people confidently insisting you are wrong about it. The more well-argued your point, the more precise and well considered, the more confident will be the folks who insist you are wrong.
I say that because I think the basic mechanism you have laid out is correct, and the assumptions you've made are perfectly fine as rule-of-thumb approximations and helpfully model what's at stake here.
But I think the question has to do with undefined variables about how the bubbling-up process would work. If the bubbling up is aggressive and the votes of the late-comers worth a lot, you're right and quality discussions will get diluted. (It's amusing to me, by the way, that people say you are wrong for neglecting these variables, and they proceed to assume the variables work in a way that's most favorable to the contrary argument).
But the other possibility is that the bubbling up is not aggressive- it's slow, the upvotes from the "deeper" community are worth the most, or the attention from outsiders is too late to substantially influence the trajectory of the discussion.
I personally think that, whatever the variables, enough of the "important" stuff that sets the trajectory of the discussion will have already happened by the time the unwashed masses get ahold of the post. So even if the newbies enter the discussion, they are entering a discussion where the terms have been set already, and have more incentive to engage at a high level.
There are still tags. Science is a huge subject with many subjects within it, so subgroups just make sense for it. For posts about cancer, it can just be posted to ~sci.biology with the tag “cancer”.
Maybe eventually we can have autocomplete tags/suggestions based on how often it gets posted which should also help keep things organized.