10
votes
How is relevance determined in comments?
When sorting comments, ‘relevance’ is one of the options. How is this determined? Is it an algorithm? If so, what factors are used in determining this?
When sorting comments, ‘relevance’ is one of the options. How is this determined? Is it an algorithm? If so, what factors are used in determining this?
You're correct about 'relevance' currently being the vote weight after labels have been applied.
Each 'Exemplary' label applied to a comment adds a 0.5 multiplier. The vote calculation =
source
From what I can tell, the most relevant pieces of code related to weighting and sorting appear here.
num_votesentry in the database for a commentnum_votesand active labelsAs indicated in the source that @Algernon_Asimov linked to in their comment, label weights will come into play as well. Presently, all users seem to have the same
comment_label_weightvalue as defined bydefault_user_comment_label_weightin a configuration file, but it is implied that at some point in the future this may change so that each user's individual weight may become fluid in response to their activity on the site.Not just implied... but also explicitly spelled out as a potential part of the eventual plan for the trust/reputation system. ;)
https://docs.tildes.net/mechanics-future#trust-based-on-consistency-and-accountability
I'm not sure how I feel about the trust based vote weighting. I feel like it would encourage people to always vote on the thing which already has lots of votes, even if they don't think it's high-quality. Equally it adds a risk factor to voting on something early. If it goes unnoticed then you've voted on something the system sees as bad and you get punished for it.
I suppose if everything is always noticed and properly voted on by the community that can't happen...
I know you didn't write the Future mechanics page (probably? no idea), but I figured I'd put it out there.
My usual way of explaining it is that it's playing with fire.
Weights too heavy, and you create power user cliques and echo chambers. Weights too light (with everyone being equal defined as none at all, the lightest) and any content the group's regulars are interested in gets washed away by new users. That's always been reddit's problem. The power user thing played out on Digg, that was their problem.
Gotta find the balance in the middle. The main advantage of vote weights is that they are power for the lurkers first and foremost. How do you get your lurkers to contribute, and how do you reward them? They do more than 90% of the voting, so it makes sense to give them some more power in their votes. Lurkers outnumber people who post by 90:1. If this is done right, the lurkers of any group would become the steady rock that keeps it true to its goals by voting on the content.
I think the trick is in keeping the weights small. Giving someone a 10:1 weight is crazy. Giving someone 1.05 is safer. The smaller the boost, the more the effect has to come from the aggregate, from lots and lots of voters, instead of from a small pool of them. I think that's the way to move forward with the weights. I imagine much larger, older groups will need larger weights than younger ones though, the problem is likely to change form as it scales.
At this point on Tildes weights don't even make sense, but the bedrock is there in the code so that on the day when we do want to start in with this stuff, the levers will be there waiting.
A well-tuned weighting system would certainly help in situations of astroturfing and brigading as well, particularly in more niche groups that might have a relatively small, but active, number of users. This is why I like the idea of weighting being based on groups rather than it being site-wide. Presumably, there could be a multiplier which scales based on active users over a period of time.
The decay is an important factor as well, since we've seen the sale of high-karma accounts on reddit being used for blatant advertising or for other mischievous purposes.
Yeah, the karma for sale thing bothers me. There's no safeguard. Sure, if someone goes inactive, the decay will bring them back to normal, but if someone decides to sell out a hierarchy mod account or something like that, it changes hands while still at peak influence. It may seem like a silly thing to buy and sell, but on a site the size of reddit, those accounts are worth five figure sums, and on a place like Tildes that'll likely be the most ad/spam resistant aggregator, it'll be worth more. That's a lot of incentive.
Noise comment, I know... but I just wanted to point out how great that statement is and how important I think the philosophy behind it is.
We are so lucky that reddit was there first and basically, did everything wrong so we could watch and figure out better ways. When I first got to reddit, I was as bad as go1dfish and freespeechwarrior. After talking about this stuff for so many years with so many people, I couldn't even explain my thinking now to my younger self. Some of this stuff you just have to see play out in the real world to believe. You can't know what moderation is like until you're doing it, you can't understand why it's so vital until you've gotten off the phone with police to handle an imminent harm situation (the suicidewatch mods can tell some hair raising tales you do not want to hear).
Now that we've been through all that, maybe, just maybe, we can finally figure out how to get a community to govern itself. I want that, badly. I want to make moderators boring, unnecessary, barely-empowered users that collectively accomplish everything we all need with a couple mouse clicks, with every user always in the running to become a mod. I want to put a bullet between the eyes of Eternal September.
I was a mod of suicidewatch, albeit only for a few years in the early days, so can absolutely verify that. It's the most depressing, scary and emotionally draining moderator job on reddit and I honestly don't know how SQLwitch has done it all these years without losing her mind. Especially since, even though reddit liked to hold SW up as a shining example of the good that was possible on the site in the early days, from my understanding, these days the support for it has all but dried up. Not to mention the policy changes that have taken place with regards to how reddit inc handles suicide threats (i.e. now refuses to, leaving it entirely on the SW mods' shoulders).
I heard something to the effect that reddit's new policy for the sw mods was "stop bothering us and handle it yourselves." I'd call the drones who made that decision cunts, but they haven't got the depth or the warmth for that. :P
Reddit has definitely made me a bit bitter. I need to work on that.
I had some tough times early on when I first start moderating, but I came out of it more jaded than bitter. "Another troll/racist/homophobe/spammer? Ho hum. <click> Next!"
As for Reddit itself, I gave up on the admins being helpful soon after I started modding /r/Help and /r/ModHelp. I realised quite quickly that the admins in those help subreddits did fuck-all to actually help people. They were just there for
showhistorical reasons. Even some of the non-admin mods were just there for the prestige of it. The only time the admins did anything in /r/Help was when they'd created an actual bug in the code or when they hadsome propagandaa message to share with the users. So I ran that sub pretty much as a one-man band while I was there. Some of the other mods (especially the few I added later) were helpful, but I did the lion's share of the work there.And then the redesign happened - and I, as a long-standing moderator of /r/Help, who was at the coalface dealing with everyone's complaints about the new system, and who had worked as a Business Analyst on software development projects for nearly a decade, was ignored. Noone paid any attention to my feedback about the changes.
I'm jaded, but not bitter. Not quite. ;)
The admins led the listentothis team on a couple times with 'sounds great, we'll look into it' that of course went nowhere. L2t needs two small, simple, easy to code changes to shine, and they won't do it. It's so little work for so much beneficial change. That's kinda why I'm ticked.
Tildes will get the benefit of everything that happened in the music groups on reddit. It's reddit's loss, Tildes' gain, and I can't wait for ~music here to start to grow into what the 800+ music subs on reddit have become. I am convinced we can turn that into the greatest music discovery and sharing engine ever, just by generating charts and playlists from all the submitted content. It's going to be breathtakingly good stuff. Tildes' tags will make short work of sorting it in any way people want to see it sorted, too.
It's funny how we carry our histories with us. You're focussed on ~music, whereas I can't wait for ~humanities to get going (I got my start on Reddit at /r/AskHistorians).
And those two different backgrounds bring very different perspectives! I don't know if you recall a conversation we had many months ago where you were talking about all the automated moderation that happens at /r/AskHistorians, and I had to disillusion you by pointing out it was mostly manual. That highlighted to me our different backgrounds.
Our different experiences on Reddit have moulded our expectations of Tildes. I'm not so much looking forward to the tagging system as I am the new moderation system, where trolls won't be able to get a foothold because the community members themselves can push them out (rather than relying on a handful of overworked and stressed moderators!).
All those perspectives are so vitally important. We want people from every forum to tell us their experiences, what worked, what didn't. All the common problems and uncommon solutions. I have faith that if we pour all of that knowledge and experience into the right crucible, we can at least take the next step towards better community self-governance. All we need is the courage to participate and the wisdom to listen.
There is a section on the page you mentioned regarding concerns, and this is certainly something that was a huge problem on Digg back in the day and with others since. I think ultimately it comes down to finding the "least worst" system and mitigating as many of thee downsides as possible, using adjustments along the way.
Yeah, there are certainly risks to such a system but I don't think those potential problem with it are insurmountable nor entirely unknown. I also don't think that docs page really does the idea justice either, since it's just a simple blurb on the potential possibilities and not what will likely be implemented in the final form.
Correct, Deimos wrote them, though I did play a small part in the original brainstorming process for the site and all the potential systems it might have, along with many others who contributed as well.
In the interests of full disclosure, there was a slack before there was Tildes. Deimos invited a bunch of us who like to talk about this stuff (theoryofreddit junkies, ex-reddit admins, grumpy burned out mods like me and cfabbro) to help brainstorm ideas. I'm pretty sure he invited me because of this old rant and the now-defunct curated content system he kindly build into automod for the listentothis team. That didn't work out so well on reddit, but the idea is kinda similar to the exemplary label, and the future exemplary upvote for bumping things up to the parent group.
We argued, he listened, some of our crazy trains of thought actually made it into the early Tildes code base - mostly the kind of things that would have been hell to add in years down the road, like core changes to the vote mechanics. That's why the Tildes code has some curious bits floating around that aren't being used yet. They are there for possible future systems. It's fair to say most of those systems came out of thinking about reddit-sized sites, which is why they are premature for Tildes right now.
I just read the rant post you linked. I love that there are communities built around building communities, and that people with actual experience are trying to build something better.
The section in the rant on velocity of a post / a type of post is interesting.
We could use tildes' article word counting feature in the calculation of how long it takes for someone to read the content. If we have length of an article we can pit it against regular text posts. Then if we can estimate reading speed (words/min) we can calculate the average time to read a post. Now all text posts share a unit with video posts, and we can compare the two. If a post is voted on way before anyone could have looked at it then its probably clickbaity.
That model doesn't handle images well. There's plenty of r/greentext posts that take longer to read than a meme-template style image. But we can't estimate the reading time, they're all just an image.
Anyways. Slight tangent there. Also I used the readability tester in the last section on this comment, and I feel super self-conscious now. It's telling me not to use the word super there. Well it feels right, so sue me.
Haha I already love this site. That thread will take me a while to read through, but having this already discussed is great.
Also, I've never heard of wallabag, can we just rip the reading time estimation out? Or I suppose just reference their implementation...
If the user can choose their reading speed we might have to put an upper limit on what a user can say their reading speed is. (or at least a limit on how their reading speed is used in the calculation for estimating clickbaity-ness)
Otherwise they could set it absurdly high and now no matter how quickly they vote it would never cross the clickbaity threshold.
That would be nice for link sharing in general! The link you send someone will actually have the content you wanted to send, even with edits to an article.
I'll add this to the list of things amp breaks though... I really wish amp would die.