Comment tags now affect sorting, more changes coming
After re-enabling comment tags a little over a week ago and starting to experiment with some effects, I'm going to be adding some more and continuing to adjust as I keep an eye on how they're being used so far.
I've just deployed an update that changes the default comment sorting method to one named "relevance" (subject to change, suggestions welcome). This mostly acts like the previous default of "most votes", but also takes into account whether comments have been tagged as certain types. As with the other tagging effects so far, these effects will probably be adjusted or may even be completely changed as we see how they work in practice, but for now:
- If multiple users tag a comment as "noise" or "off-topic", it will be sorted below comments without those tags. That is, comments that are not noise or off-topic will be prioritized above off-topic ones, and off-topic will be above noise.
- In addition, comments tagged as "joke" will act as though their vote count is halved. This will just help with de-emphasizing joke comments a bit for now, but I definitely still plan to have filtering/collapsing behavior attached to them eventually
- The "troll" and "flame" tags still don't have any inherent functionality yet, but I've been using them a little like a reporting function in the background so far, so those tags are helpful to me for pointing out comments that may need attention.
Let me know what you think of these changes or if you notice anywhere that they seem to be working poorly. There should be more updates and changes to the comment-tagging system coming this week as well, based on suggestions and observations so far.
I wonder what sort of voting pattern this change might lead to. Currently the first semi-popular top-level comment tends to stick around at the top. Even for multiple days as some threads are more sticky than others.
Will this lead to more of a treadmill effect as these sticky comments are more likely to be seen, and thus more likely to be tagged? It might allow more variety at the top.
I also like how Hacker News does it, where brand new comments get a few minutes at the top before being moved to their natural place. It means good comments are more likely to be given a chance, and bad comments are more likely to be downmodded/flagged.
Of course, it also means bad comments are potentially more visible.
Yeah, one of the other reasons I wanted a new comment sorting method with a vaguer name was to be able to eventually experiment with things like that as well. I think the idea of putting new comments at the top for a little bit to try to give them some initial exposure is interesting and worth trying out at some point.
Would you track the number of times a comment has been in number one pole position?
It might be interesting to use the number of times a comment was "at the top and not voted on" as a negative vote when calculating the lower bound of Wilson.
By "not voted on" I mean someone voted on another comment that was lower down.
Will "noise" and "off-topic" keep the same weighing values for the foreseeable future or will one essentially be an alias for the other?
Will other sorting options remain unaffected and is there room for sorting options which would prioritize, for example, "joke" tagged comments?
Is the plan still to restrict comment tagging to users with a certain degree of "trust" and if so, will that mean access to tags globally or will tags with similar functions be available for different levels of trust?
Maybe I'm misunderstanding what you mean, but they're not the same currently. Right now, off-topic gets put into a higher "tier" than noise, so if you had 3 comments where one has no tags, one is off-topic, and one is noise, that's the order they'd be displayed in, regardless of the number of votes on each (more or less, see below).
There are also currently two other differences with the noise tag (which again, may be adjusted or changed as I keep monitoring):
Yes, I'd prefer to keep the sorting options as accurate descriptions. I want "sorted by most votes" to mean "sorted by most votes", not "most votes*" where there are a bunch of exceptions. And yeah, I definitely think that sorting/filtering methods to see things like no jokes or only jokes would be a good approach and something I'd like to work towards.
Yes, I'd still like to make the trust system a core part of the comment tags in different ways. I'm not sure exactly how it will end up working, but there's also the possibility to adjust different users' "tag weight" that lets us trust their tagging more or less without going all the way to a binary "you have access to tag things or you don't". Right now, I have everyone's tag weight set at 0.5, which at least for now, mostly just means that 2 people need to tag something the same way for there to be any effect. But in the future we could do things like start people out at a lower value and see if they seem to be tagging "properly", then gradually adjust it upwards (or lower it even more if they aren't).
Couldn't this also be the case for offtopic or troll tags as well though?
In some ways, but it's not nearly as clear. It's perfectly possible for a comment to be both off-topic and a very good comment, so the votes aren't contradicting the fact that it's off-topic. It's a good comment that a lot of people voted on, but it's off-topic.
However, it's not really possible for something to be noise (which implies that it doesn't add anything to the discussion) and also be highly-voted (which, at least in theory, means that a lot of people think others should read the comment). In that case, votes and the tag should be expressing things that are in direct opposition to each other. You can't really say "this comment doesn't add anything, but it's a really good comment". It has to be one or the other, so the cancellation behavior makes more sense.
Reddit is plagued with highly upvoted noise; so just because the masses vote for something doesn't mean it's worthy. That won't be a problem right now when the ratio of people who care about quality over inanity is so good, but when you open the floodgates and September hits that could be another matter.
Thinking out loud about a few different scenarios involving the noise tag and voting:
Someone makes a noise comment. It gets caught and tagged early. Because it gets sorted to the bottom and collapsed it doesn't get more votes. Tagging worked and the comment is dead and it only took a few taggers. The knights of the new are critical here. Perhaps acting early should award more trust.
A noise comment doesn't get caught until it is highly voted. Some users tag it as noise but it takes a lot more to fight the tide of existing votes. This is down to the ratio of users who care about quality to those who don't, or more accurately the ratio of the former's tagging weight to the latter's voting weight. Giving the right users more weight will help here.
A well written comment expresses an opinion that ruffles some feathers. Some users who don't like it see it early and mark it as noise to suppress it. It gets collapsed. Most others don't see it so it doesn't get voted out of its collapsed state. The abusive taggers may or may not get caught and penalized. Again the first on the scene have an out sized effect. Catching and penalizing abusers will be important.
A similar comment to 3 gets made and highly voted. Those that don't like it form a brigade and mass tag it. It's fairly obvious so the brigaders are easy to punish.
Distinguishing between the first two and the latter two will be important. Is this really noise (highly voted or not), or is it a quality comment that someone has a grudge against? That will take human review. Something, something, who watches the watchers.
Kinda sorta 'controversial' sorting?
Will we have a way to change our default comment sorting? I appreciate what is being done here, but for now I'd rather have raw votes. I don't think I'm seeing so many jokes that I'd want them artificially bumped down, for example.
Probably eventually, but not yet. It makes it a lot more difficult to figure out why certain adjustments aren't working well if the people that don't like them just stop using the sorting method entirely instead of giving feedback about why they don't like it (and finding examples of it being a problem).
Comment threads are generally still very small and tags are rare, it's unlikely that this is going to impact you much at all yet.
This, ladies and gentlemen and others, should serve as a reminder that this is a website in alpha-testing, and we are all merely Deimos' toys. :)
What's exactly the difference between “noise” and “off-topic”? And between “troll” and “flame”?
And, we need a "fluff" tag for your topic. :)
@Deimos explained that to me over here.
Trolling is when someone tries to disrupt a thread by posting controversial material just for the sake of causing trouble (compared to holding unpopular opinions). Flaming is when someone posts direct personal attacks on other people.
I see, I thought flaming was like a more intense trolling. If that makes any sense. I now have to go find one mistagged comment lord knows where... Thanks.
While we're at it, you seem to always come to the rescue whenever I have a question, so: do you know why can't we tag our own comments? @Algernon_Asimov
Because Deimos didn't give us that ability. :)
The tags are there to identify negative comments. As Deimos wrote when he re-enabled comment tags: "If people see that their comment is getting tagged negatively". Comment tags are for other people to categorise your comments as negative. If you know your own comment is negative... why would you post it in the first place?
Of course, in reality, it's not quite as black-and-white as that, but the overall philosophy is that it's up to other people to decide whether your comment is a positive, neutral, or negative contribution to the discussion: whether they should vote for it, tag it as off-topic, or tag it as noise. You don't vote on your own comments, and you don't tag your own comments, because you're not impartial.
Yet. It's entirely possible some of the tags (the more neutral ones, at least) will eventually be allowed to be applied by users to their own comments, e.g. Off-topic, joke and perhaps other ones that might get added eventually like sarcasm, etc.
I also think your labeling all tags as "negative" requires understanding the context in which they are being called "negative". It doesn't necessarily mean those comments shouldn't be allowed or people shouldn't ever make any comments that have the potential to be tagged. IMO "negative" simply means those comments negatively effect the signal to noise ratio of the comments section with regards to on-topic discussion. Saying "thanks" to someone who provides an answer to your question is "noise" but not something people should be forbidden or even discouraged from doing. And even off-topic discussion can still be incredibly valuable... it's just potentially less valuable than on-topic discussion, so it generally helps to distinguish it, is all.
I was just thinking about his reply and wanted to come back to argue that not all noise or off-topic replies were negative. But you worded it quite nicely already.
Yeah, I do hope @Deimos allows us to implement tags on our own comments. I want to be able to thank someone and tag that comment as noise. Or discuss some typos within an off-topic tag.
Then is it wrong for me to tag your post here as a joke? Because, while it's not something I'd tag negatively, simply because I believe it adds a little levity to the conversation, it is a joke, with no other real "value" than levity.
For the record, I both voted for the comment, and tagged it
Joke, the former because it's worthy of being seen, the latter because it is, in fact, a joke.No.
You should also tag this comment of mine as a joke.
I see. Thank you.
As explained above noise is something that doesn't add to the discussion, or even distracts from it with no meaningful gain. Off-topic is something which can still be interesting, but is not pertinent to the discussion/subject at hand.
Noise is "This."
Off-topic is "Hey I noticed you mentioned this university, did you go there?" on a topic about the diets of Mediterranean plants.
But there's some overlap between the two, right? Won't most off-topic comments be noise?
Arguably there's some overlap, but the difference is which comments you judge a net negative to the discussion (noise) and which one you judge to be a net positive but still technically off-topic.
Actually, Deimos explained "noise" to me as "the presence of this comment makes no difference to the discussion", rather than it being a nett negative.
I'm not sure I agree with the 'Joke' tag having such a negative impact. A good joke can often be insightful in ways that a straight reply cannot be (see satirical comedians like George Carlin). Though I understand the vast majority won't be at that level, I think a halving of votes is excessive.
I think the intention there is to really separate the bad jokes from the insightful jokes. If someone makes a really insightful joke, the votes will still help to bring it up. The insightless jokes then will be put much further down, where they will likely receive little commendation.
I could see halving votes being too much, but I'm not sure. If comments end up being organized linearly (ex: 1st has 1000, 2nd has 900, 3rd has 800, etc) then you could bring an insightful joke from 10,000 to 5,000, which would considerably hinder it. However, if comments are organized logarithmically (ex: 1st has 1000, 2nd has 500, 3rd has 250) then you would only be bringing a comment down one or two notches. Depending on that order, it may be highly inconsequential.
Oh I understand and agree with the intention, without a doubt, but some joking comment could have the most votes in a thread because it is both humours AND insightful, but enough people mark it as a joke (I don't know what the threshold of tag -> action is, @Deimos?) then it gets bumped way down in visibility.
I read somewhere that the tags are to be used specifically for insightful comments, and thus a 'good' joke should not be tagged with 'joke'... I'm not certain on this aspect though, and could bring it back to the tag -> action threshold...
Maybe a better way would be the ratio of joke-tags:votes a comment has. That is a comment has been tagged as a joke more than it has been voted, then perceived vote could be halved for thread-sorting.
It may very well be excessive. Like I said, the behavior for "joke" will probably move more towards something filtering-related instead of sorting-related eventually. For now, jokes are rare and threads are small, their visibility isn't going to be particularly impacted.
I'm afraid people might start using tags as downvotes, especially now that they affect sorting.
Changes seem good as described. Trying to run through some existing posts to see how they'll look, but not too many have enough comments with tags to really say beyond "it's functional".
Any suggested topics to check out? Otherwise, just wait and see I guess...
I created a ~test thread to test comment tags: https://tildes.net/~test/6hv/comment_tag_testing
Since I can't tag my own comments I need at least two other people to tag the comments, though.
Cool, checking it out...
Hmm, I don't know if there are any topics that are particularly good demonstrations. The tagging is still pretty sporadic, and a lot of the off-topic tags seem to happen a bit "deeper" inside comment threads where the sorting effect might not be very significant—it would just push the comment down below "siblings" in that branch (if there even are any).
Some thoughts.
With the additional effect of collapsing comments with 2 or more noise tags this is functionally like downvoting. It will be used in that way regardless of original intent.
Fair enough, I'd like to see what happens due to this.
Just....be careful how long you let this go on. Tildes is small right now but will likely get bigger quickly, easy to get overwhelmed by the negativity on your own.