Bias towards older comments
Older comments have an unfair advantage on Tildes if you sort by votes: they have had more time to collect votes.
What's interesting is that Reddit is less affected by this problem: since the default sort is "best", which sorts by expected (in a statistical sense) upvote/downvote ratio, newer comments with a good ratio can quickly move to the top.
I don't see a straightforward way to extend this to Tildes, since we don't have downvotes. Any ideas? Of course you can sort by newest first, but then you lose the benefit of votes entirely.
Maybe we could compute the expected final number of votes, based on age, current score, and a model of how comments gather votes as they age? Is there a way to download tildes data somewhere? I could try to investigate.
Unfortunately, this is basically a problem with no real solution. Any method has to give some comments more visibility than others, which makes them more likely to attract more votes. Anything that's actually "fair" (totally randomizing the order, only showing a subset of comments to random users) ruins the point of even having a voting system (or even a commenting system, in some cases).
The "best" method on reddit has some benefits, but it also works poorly in different ways. If you go to a large thread in /r/AskReddit or somewhere similar, pay attention to how far you have to scroll down before you see the actual best comment. Often, a comment with a way higher score that's quite a bit higher quality than the other comments will be very far down the thread, because the "best" method is tilting things too much away from votes.
In the future, I hope that maybe some systems like the comment tagging and "exemplary votes" (to help push truly great comments up faster) will help, but there will always be biases because of the "more visibility = more votes = more visibility" feedback loop inherent in the system.
How about a concept of "fresh" comment sorting that is a mix of age, replies and votes. A brand new comment will be at or near the top but will rapidly fall if there is no discussion or votes on the comment. If a comment is highly discussed or voted on the thread will remain fresher for longer.
It would probably work well for some things, but poorly for others. There were would be some similar biases to what we're seeing with the overall "activity" sort for topics—controversial posts that attract more replies or cause stretched-out arguments get more attention.
In the end, it'll probably end up with a similar issue in a slightly different way. Instead of "the comment at the top attracts more votes which keeps it at the top", it would be "the thread at the top attracts more replies which keeps it at the top". It would definitely be better for new comments than starting them at the bottom though (which is what happens now).
Perhaps just having more options on how to sort votes is the solution. No option is perfect, so letting users choose between different sorting means could give more flexibility to help address different kinds of topics and discussions.
Of course, whichever is the default will by far be the most prevalent. And as userbase grows, the default options become so prevalent as to be essentially the only option.
I like this idea. Gives a chance for previous comments that have already contributed to the discussion in both what they say and the responses they generate to be discussed further, whilst also giving new ideas that initial 'bump' that allows the point to be heard. I guess the only thing that would need fine-tuning would be the freshness calculation of the comment and what the algorithm would prioritise. A funny comment could receive lots of votes but no replies, would this be enough for it to stay fresh, and if so it may be placed above comments that contribute more to the discussion .
Is it possible to set the placement of comments based on
age of comment / votesor something along those lines? If so, it'd be nice to have the top spot chosen randomly between themost votes(1) andnew active(total new comments with a similar average voting time to the current top comment, with all applicable comments having a sum of 1 weight.)Does that make sense?
Reddit is by no means less affected by that problem. Rather, the illusion it isn't is stronger.
In general, there's little point discussing how to hack away (pun intended) that bias before the devs offer a more concrete explanation on what is already implemented, and what their future plans are in regards to comment sorting.
Not sure, but evilnight seems to be the go to guy on this front. He clarified the concept two years ago, and one of his posts hints some form of vote weight is already at work. However, I'm not sure whether it actually affects comments.
In other words, to the system the value of the sum of votes is not the same as the number of votes. This kind of system requires time, a larger and a more diverse user base to work properly.
Could you explain why you think my argument isn't valid? Here's an explanation of reddit's "best" system: http://www.evanmiller.org/how-not-to-sort-by-average-rating.html
First of all, best requires downvotes so this won't work here.
What that method does is calculating the likely progress of the discrepancy of up and downvotes. Hence, newer posts have a chance to rise up no matter when they were posted.
Unfortunately, this concept still requires people to look through all comments, revisit that thread, and judge based on qualitative contribution rather than rapture. Impossible with the number of comments a thread can get.
Meanwhile, people actually vote notably less on any secondary branch - I'm pretty sure, if mobile didn't become popular, most users still wouldn't know they could minimize any branch -, so you can barely ever rely on the quality of posts placed at the top. Due to that prevalent culture, even low-traffic threads are affected.
HN follows a different approach with more focus on time-based decay, more similar to what hot does/did on Reddit. It works better and you can't downvote until you've got 500 karma, but its more serious and smaller community has an even greater role for its success.
So, of course, Reddit threads would be worse without the best sorting, but user behavior still dominates the outcome.
This also compounds the problem where initial votes seriously affect how a comment or post preforms. Obviously it's worse on reddit with the ability to downvote, but reddit really feels like a crapshoot in terms of how your comment will perform.
older comments are more likely to be seen at all on reddit than newer ones. newer comments start at the bottom, and the majority of voting users will have already seen the topic and passed on through.
We're using a pretty basic ranking here now - the same one reddit has been using for a decade. It definitely needs to evolve. It's not a hard problem to solve, though - once you accept the basic idea that you don't have to show every user the same view of the page/content, or even show the same user the same view on refresh. Ideally you need to mix things up. This goes for new submissions as well once a place becomes busy enough. Toss a few new comments/submissions at the top of the page. It's the only way to get fresh eyes on new content and lessen this age-advantage.
I expect we're going to have lots of comment sorting options - based on weight, time/speed votes are acquired, age and turnover (particularly important for multiple-day old threads and very busy threads), we might even have different comment sections based on where the submission acquires them as it bubbles up the hierarchy. We'll probably have a default sort that reflects the best mixed in with the newest, and then the option to click and change the view to other ways of looking at it.
It's not that there aren't solutions to the problem you describe - it's that nobody really gives a damn about it. Building a better comment framework is a challenge. Reddit won't bother for something that's going to fall of the page in 24 hours - there's no value in it for them. The only reason reddit changed their system to 'best' was to make it easier for anything controversial to get buried, as a way to keep the shitty comments and hate speech away from the top of a thread. It worked, but it also buries dissenting opinions, so in a way they shot themselves in the foot at the same time.
Once we have comment tagging up and running (and evolved beyond the basic joke/flame/troll structure) the tags might also provide some levers to use sorting comments effectively. They don't all have to be negative or neutral tags. We could even have 'exemplary' comment voting mechanics just like we wanted for submissions - highlight the best. Curators and editors could even build out the comments manually, which would make sense in places like /r/askhistorians or other strictly-controlled discussion communities.
Since we're planning to have much longer-lived threads here, tildes is going to need systems like that to support keeping the discussion healthy and giving it some sense of order and discoverability. I wonder what options might exist beyond the basic nested-comment view. Are there other ways to put a discussion together? Has anyone explored that idea? Are the other commenting systems we can look at that aren't so focused on a time-centric structure?
I had the idea (somewhere in one of those daily discussion threads) that it would be nice to be able to "move" specific threads from one place to another.
First let's look at a simple comment chain
You have a comment chain, where the 5th level down is really interesting, but many people won't get to see it, or will just overlook it entirely.
What if we (either trusted users or with enough votes by tagging) were to be able to move the 5th level comment to be it's own top level comment? We could even call it something like
cross-linking.I can see two main scenarios for this:
the comment is very interesting and should be able to draw more eyeballs, so it's linked as a top level comment within the same discussion
the comment is very interesting and also covers a topic that some users want to see, but aren't subscribed to the topic that the comment was posted in, so it is linked in that other group
Let's look at the second scenario. Let's say our 5th level comment from before was in ~music.talk and was in a discussion about songbirds. Perhaps, some people in ~science.talk would also be interested in that discussion, but aren't subscribed to ~music.talk, so a trusted user cross-links, or if a high enough percentage of people tag it
scienceit is posted in ~science.talk as if it were an original post in ~science.talk.It could look something like:
I don't know if the idea is feasible or even desirable, or different enough to warrant using. It does feel a bit like reddit's crossposting (not that that's a bad thing), but could be used to move less-seen comments to other places where they could generate even more interesting discussion.
The most obvious issue I can see with that is that a comment 5 levels down might be really good, but it often needs the context of several comments above it to make sense.
You're right.
Maybe something that also moves the whole comment chain, while highlighting the exemplary comment, is necessary.
Would that work better, in your opinion?
I don’t know. It could, but it also might just shove 4 bad comments into someone’s face so they can read one good one.
Well, then maybe it should not be done algorithmically, but by trusted users, or the original poster of the comment.
Let's say they get the option to pick "cross-link" after a certain vote threshold, then they can pick how many posts to link with context. So, they could pick 1, 2, 3, 4 or however many they want, depending upon quality.
Does that work better?
It could work. I have no idea if it would work well, but it sounds like an interesting thing to try, if nothing else.
Reddit still has that issue. If you comment on a popular post that's at least 6 hours old it's highly unlikely you'll get more than a couple upvotes.
Older comments on Reddit are definitely favored. Try posting nonsense into an early thread. You'll easily get 20% of the OPs count.
I'm not a fan of solutions that sound like an open problem in data science and/or machine learning.
What if top comments were somehow mixed with new comments? A special sorting method could alternate them or (more hypothetically) show them in two columns.
I think that Tildes is less affected by this than reddit, because when you comment on an old Tildes post it gets bumped to the top of the activity sort. Once there, previous visitors to the thread can re-enter and just scroll to the red stuff.
What if you could sort by a votes/minute metric? Do you think this would address your concern?