Tildes and multi-dimensional weighted votes
hello,
I've been wondering a little bit about what a well-designed voting system on a website like reddit or Tildes would look like, and as I do not have a definitive answer, I do have a suggestion to make. I've originally posted this on another website, but I thought that it could also be fruitful to discuss this here, seeing that efforts have already been made in that direction (similar features have even already been implemented).
Looking forward to reading you!
I was wondering whether Aether should support downvotes or not, seeing that they are often misused on other discussion platforms to suppress content that is disliked rather than non-contributory or low-quality. People may then not view content that was heavily downvoted, even though it may have been high-quality.
Should we rather use some other mechanism to serve that function? If so, what would it be?
Personally, I'd suggest that we experiment with two-dimensional weighted voting.
In a word, it would allow users to express both whether they agree or disagree with (alternatively, like or dislike) a piece of content (and how strongly so) and whether they think that that piece of content is high-quality or low-quality (and how strongly so).
In practice, it could look like this (for users): upon clicking on the voting icon, a square with two scaled axes would appear. One for the quality of content, the other for the level of (dis)agreeance. A user, who had for instance found a piece of content to be very high-quality, but who somewhat disagreed with it, could then express that opinion by click and dragging right to the top of the square, but somewhat left of its center.
That simple mechanism would therefore allow us to distinguish between those two criteria and better capture the intention behind a vote, and help alleviate the issue of seeing deeply unpopular content being buried despite its high quality. It would also allow users to express how strongly they feel about a piece of content by letting them adjust the weight of their vote. Plus, it wouldn't be too cumbersome to use (in my opinion).
(Voting strongly should be slightly inconvenient or cumbersome to do, so as to deter users from voting strongly every time, thereby rendering strong votes meaningless. In practice, that could mean having to move one's mouse only a little for a soft vote, but more and more as the vote gains more weight.
Axes should also be sticky, so as to make it easy to vote with respect to one criterion only (we shouldn't need to try and aim precisely).)
We could also put in place some additional mechanism to let users rate content with regard to other criteria (how informative it is, or impressive, exciting, funny, etc.). I do not expect users to rate all the content they read, but allowing them to do so could still be useful. They may still bother to do it for content they find especially informative, impressive, etc., and that would then allow other users to sort content with regard to one or several of these criteria and find content tailored to their interests.
(We could then also display for any piece of content a chart (that could look like this) showing how it was rated with regard to all these criteria. That's not really important, but I find that cool.
We could also plot the number of votes as a function of vote strength, find the average vote strength and so on... That would also be cool, and interesting.)
What do you think?
The kicker is your audience.
Joe is sitting in a bus terminal on his cellphone waiting for his ride. Mary is on a laptop sitting in her van about to make a house sale when her clients show up. Burt is in his dorm at college between classes. Spike is drunk off his ass at noon reading from the bar. Jane is in the dentist's office waiting room, Eddie is riding on the subway. Only Alex is browsing from his second monitor while compiling code projects at work. You get the idea - most of your audience is going to be non-technical, not geeky, and merely browsing for distraction. Somehow you have to turn all of that barely-attention into useful work sorting the site's content.
Most of these folks have no interest in rules and fancy mechanisms. The sad truth is the further away from one-click simplicity and intuitive rules you go, the fewer people will understand and use the mechanisms the way they were intended to be used. I suspect that drop off is wickedly steep, too - even a little complexity loses a lot of work. If you click on vote and are prompted with even a single sub-menu, you're going to lose a lot of people right there, just because it's different than what they've been trained to use by every other website since the earliest days of eternal september.
Reddit's downvotes were supposed to be used as crowdsourced moderation, and even that dirt simple idea couldn't survive contact with reality without downvotes becoming a 'fuck your opinion' button. One also has to consider what the users will do with any tech/tools you give them, and it will never be what you expect.
Better to put something beside the vote for when people feel like voting isn't enough - which is where labels came from. I'm also partial to the idea of having special classes of threads for polling or community content aggregation or major ongoing events or perhaps even some attempt at genuine democratic quorums for decision making threads. Rather than trying to cram all of the possibilities into a single thread class with a single interface, it's probably better to keep them separated with their own mechanisms. That should keep clutter to a minimum.
Most people have pretty neutral reactions to content. Maybe they vote, maybe they don't, they cast the vote before their conscious mind even realizes they've done it, like a reflex. I think it's important to have other options there for when the content provokes a strong reaction and capture whatever that reaction was. Labels take more clicks to use and some require entering a reason - the strong reaction is what helps the user power through and use them. Later on when the trust system matures, votes will have weights and stop being equal - that's probably the simplest/safest way to make them more useful. Adding extra steps to the voting itself is just getting in the way and will reduce the number of votes being cast.
That's not to say there's no merit to having an agree/disagree axis. Whatever merit it may have is going to be difficult to collect without breaking the simplicity. This is an interface design challenge. One of the hardest things about being a geek who gives a damn about community management mechanisms is realizing that just by even knowing what that concept is, you are light years away from your average user in how you think about the problem and in predicting how regular people will react to and utilize any given toolset.
I agree with you. When a make a new post on Tildes, I usually take a moment to think of the right tags for it, and it does sometimes feel like too much work.
With that said, would the first mechanism I've proposed (the square with two axes) really be too burdensome to use? Or would we rather get used to it after a while?
It may actually not be too much work: it's still a one-click mechanism. Fair enough, it's not just click, rather click-drag-and-release, but that's it! You draw an arrow and you're done.
You may even make it a habit to draw a small upward arrow as a generic response, an equivalent to our mindless votes.
But when a post deserves better than our inattention, when its high-quality calls for stronger support, then you're equipped!
Is it very high-quality and you strongly agree with it? Fully veer to the top right corner. Is a comment well-written, but you really disagree with it? Just sway slightly upwards, and move fully to the left.
It is simple to understand, and simple to use. It does require a little bit of thought–but not that much either.
And after all, Tildistas tend to be the intelligent and cultivated kind of folks, it seems to me.
The kind of folks whom I could see use such a mechanism :)
Who knows if we won't actually like it once it is in beta-testing?
I think we should give it a try and see how it goes.
As someone who works with design regularly, I can guarantee you that a change like this would cause a ton of confusion. Anything that you have to teach someone how to do is bound to fail unless it's absolutely necessary for the product the function.
The idea of a "vote" and a "high quality content" already exist in the form of the vote button and the label "exemplary".
While I applaud your efforts to improve the website, I think perhaps the best first step is to better understand the problem rather than searching for a solution. Because the problem isn't that multi-dimensional weighted votes don't exist (they do, as noted above), the problem is something else and we shouldn't be trying to create a solution until we understand the parameters of the problem.
This is a valid concern, I see where you're coming from.
I am not sure though, whether this feature would really need to be explained to Tildes' users... On the one hand, it is very intuitive and easily understood: I agree, I go up; I like, I go right... and on the other hand, Tildes' userbase consists of a certain kind of demographic, a kind that wouldn't struggle to use such a feature...
The best way to judge, in my opinion, would be to make an experiment and look at the data. What does it tell us? Do users effortlessly adopt that new feature? Is it confusing? If it is, can we tweak the feature to make it less so? Or is it straightforward enough to use? Do users vote more? Less?
I understand that we humans are averse to change, and that has its advantages, but sometimes good things happen when we change!
:)
Again I'd push back - what problem are we solving?
I can't tell you how often people think "more is better" or "different is better" without considering what problem they were trying to solve with new data or a new design.
You have to consider that a significant amount of work needs to be done before this can be accomplished - the graphics of this have to be thought out. What about mobile users? Users who don't use mice? Then you have to think about how to put this into production, but for only a subset of users so that you can collect data and compare how it's being adopted. It's not as simple as "turning it on".
Because it's not that simple, there's a very good reason why entire processes have been built around tackling the problem before trying to implement a solution.
Probably not for pc users, but it's really mobile unfriendly. Dragging my finger to a specific part in a square (which as a whole will not be bigger than my thumb) that represents my opinion of a comment is not very practical. Also how is the voting distribution gonna be represented? A heat map of the tiles? How is this square even gonna be divided? Is it just gonna be a square where you click/tap over where you think the comment belongs? Is it gonna be divided in a grid? If so, how many tiles will be in that grid? Also, having an axis that says agree/disagree is probably more reflective of the voter than the comment.
I understand, but I am not so sure that it would be difficult to use...
There are some games on mobile in which one can control a character and move around in a world in three dimensions, like this one.
Do you see that circle in the bottom left corner? The square I've proposed could (but doesn't have to) look like this circle. I don't play a lot on mobile, but I reckon that one may still easily express their opinion with the square I've proposed, seeing that one may already move around in a 3D world using a similar mechanism.
I'd also like to stress that the square could probably be designed in a way that is not too invasive for the user (so that it doesn't become clutter on the screen).
As for the voting distribution, I was thinking of a traditional plot using two axes. We would then draw a curve for the quality of the post, and another one for the level of (dis)agreeance. We would have more data telling a more nuanced story. Voting wouldn't be binary anymore, and we'd observe votes being distributed around a mean value. On some posts, it would be quite dispersed around it, on others, much less so, the mean value would differ, and all of this would better tell us what posts are of higher quality, what posts are more controversial, what posts are more low-effort, etc.
In my opinion, that'd be great.
(I'm sorry I haven't answered all your questions, I have to go somewhere and I'm running late... see you around!)
Maybe this is just me being stringent and imagining your button as something else, but that's pretty large for a vote button. Also there isn't a lot of room in tildes's UI for that, atleast for comments.
So it's gonna look like this? (Horribly made ,i'm sorry.)
I need to learn proper formatting
(No. Of votes are the dots)
Agree <---> disagree
.. ... .... .. . HQ
⬛⬛⬛⬛⬛ ..
⬛⬛⬛⬛⬜ ...
⬛⬛⬛⬜⬜ .....
⬛⬛⬜⬜⬜ ..
⬛⬜⬜⬜⬜ LQ
There is possibly a simpler way to measure this in the current interface. Clicking the same vote button on the same comment multiple times could iterate your own vote to represent a higher level of agreement. People do love to tap on things repeatedly - remember that ancient reddit april fools experiment where they let everyone on the site vote on the same links multiple times until the score hit infinity?
That's the simplest way to do this sort of thing I can think of.
It's not exactly elegant but it does sidestep some of the UI issues.
Absolutely. A 2-D graph is very technical for users voting on whether they like a post on a forum. I think even a 1-dimensional control would be too difficult for most users. Most users are used to at most upvote/downvote. For many years, Facebook didn't even have a thumbs down. It was only thumbs up or nothing. I'm sure they did the A/B testing on this and decided it was better for their use-case. When they did eventually change it, rather than a sliding scale of any sort, they basically made it an enumeration. I think the options now are something like "Like," "Laugh," "Surprise," "Heart," and "Dislike" or something along those lines. It's not even a 1-dimensional scale. It's just a bag of options. I think the labelling system here looks like it solves the problem you're trying to solve with this new control.
I'll say the bigger problem is that for a week I couldn't even figure out how to vote on a topic. I could vote on individual responses, but it took me a week to realize I could click on a story's current score to increase it. That's very hidden in my opinion.
Love the theory but I doubt people would bother to read the manual.
I generally agree that it's frustrating that nobody seems to bother to treat the "upvote" as a vote for quality, rather than opinion, amusement and whatnot. But adding further complication probably wouldn't make this any better.
My armchair theory has long been that votes could be entirely replaced with tags that spell out what you're labeling the post as. Humorous, insightful, hateful, etc. I think Tildes treats some of the comment tags like votes? Or it was at least a plan? But generic votes likely still drown out any tags.
iirc noise and joke labels push those comments further down the page and/or auto collapse them. This seems to be working in my experience.
All but "Malice" effect the comment sorting in different ways, but only "Noise" collapses the comment. You can read about specifically what all the labels do in the Tildes docs:
https://docs.tildes.net/instructions/commenting-on-tildes#labelling-comments
edit: Worth mentioning is also that "Noise", "Offtopic" and "Malice" also flag comments as "uninteresting" and prevent them (and their descendants) from bumping topics in the "Activity" sort as well... though I don't think that's mentioned anywhere in the docs yet.
You can also check out the relevant bits of code behind them all, if you're so inclined:
https://gitlab.com/tildes/tildes/blob/master/tildes/tildes/models/comment/comment.py
https://gitlab.com/tildes/tildes/blob/master/tildes/tildes/models/comment/comment_tree.py
https://gitlab.com/tildes/tildes/blob/master/tildes/consumers/topic_interesting_activity_updater.py
This reminds me: is it possible to de-apply the labels that cause de-prioritization of a comment? Suppose a user acting in bad faith goes on a labelling spree, for a user, a topic, or a particular set of comments. Could this be undone by users should they detect foul play?
At least for Noise, yes, that is possible; Other users can counteract the effects simply by voting on the comment in question. See:
https://gitlab.com/tildes/tildes/blob/master/tildes/tildes/models/comment/comment.py#L319
(Note: AFAIK every user's label weight is currently set to 1.0, but that may change in the future, especially as the trust system gets developed)
https://gitlab.com/tildes/tildes/blob/master/tildes/development.ini#L39
But AFAIK, for all the other labels, no, users cannot currently undo the effects directly... although, since they are only minor adjustments to the sort position they can essentially be counteracted just by votes as well.
p.s. I wouldn't recommend anyone go on a comment labeling abuse spree, as that is potentially grounds for having your label privileges revoked, or perhaps even being banned.
Would be interesting how much those are used. They’re no longer displayed which makes sense but I tend to plain forget they exist (which might be partly because of the comment quality being pretty high on Tildes and most of them being negative — more positive labels beyond the somewhat extreme “exemplary” would be nice!).
It's probably worth linking the original threads on that feature: 1 | 2 | 3 | 4
These are all simple stubs for what will someday grow into a more nuanced toolset. Unique labels for each ~group to help with their own content, for example. Labels also don't show up until one's account is a week old, so that people have time to acclimate to the site culture before being tempted to use them... and to prevent abusers coming back in on new accounts from using them.
Expect something like this for threads themselves someday, too - once the place is busy enough for there to be real competition between threads. Deimos has been building Tildes with these ideas in mind even though they are far from being mature implementations.
The RoosterTeeth forums back in the day had a pretty granular system. I detailed it here.
It wasn't used for any sort of curation or moderation, it was more of a social-status/karma function. It actually worked decently well by approximating a general sense of one's "reputation" on the site. But the mechanics of interaction on a bulletin board style forum are different from something like Tildes or Reddit so it may not map one to one.
?
Overall, it's tough to overengineer/overspecify moderation systems. Even Netflix went from a 5 star system to a thumbs up/down system because people are averse to putting too much thought into decisions. I think the "fun factor" of the Roosterteeth model was good and intuitive enough to make people participate in it. But the abuse was still there. People would still organize mod-bombing to neg posts they didn't like. Even though it didn't filter by negative votes at all, just having massive numbers of negative votes feels bad and has a chilling effect on alternative perspectives. It also tends to make people more combative since it feels like an "attack."
Complex voting systems also have an issue with bias. The people who engage with complex modes of engagement might not be representative of the average user, so you can get a divergence in who does the curation from who is doing most of the reading.
How so? Aren't they basically the same thing, mechanically? You post a topic, others comment on it; people vote based on the criteria they put before themselves; repeat ad infinitum.
Bulletin boards are organized around threads. One thread can cover multiple topics and posts go up in chronological order with no nesting. In other words, all posts in a thread are a 1 to many conversation. Multiple parallel conversations can happen in the same thread, but they go chronologically so they all end up weaving together as participants engage.
In a format like Reddit or Tildes, the board is organized around links. Each thread generally only cover the topic of one link and conversations are threaded underneath it so the conversation ends up sprawling across multiple independent branches under the same topic. In other words, communication is a bunch of independent 1 to 1 conversations rather than posting to the group. And, if you're doing vote-based curation, then the sort ends up being affected by vote count or whatever.
On a subjective level, the vibe in a bulletin board feels like having a conversation with a big group of people around a large table. Everyone is involved and participating at once. The vibe in a place like this seems more like having a bunch of simultaneous conversations with people.
What you're proposing is a more rigorous Slashdot approach to voting.
Like @notis said, I don't think that most people would bother with more than the like/dislike aspect of it.
★✰✰✰✰
I like this idea very much and I think it would suit the site very well but I misclicked like all those amazon reviews.