45 votes

Steam updates user reviews with a new helpfulness system

18 comments

  1. [2]
    BeardyHat
    Link
    Good move on Valves part. I often look at reviews before I purchase and often it means sorting through a bunch of meme garbage before I can get to the meat. It's fairly similar to what Reddit had...

    Good move on Valves part.

    I often look at reviews before I purchase and often it means sorting through a bunch of meme garbage before I can get to the meat. It's fairly similar to what Reddit had become, where you'd go into a comment section for more info and it would be a string of puns or jokes upvoted to the top.

    I need less low effort garbage in my life, not more, so I'm happy for this change.

    56 votes
    1. elight
      Link Parent
      "I need less low effort". You mean people giving low effort? Because I want to do low effort when it comes to decisions. My brain hurts enough already!

      "I need less low effort". You mean people giving low effort? Because I want to do low effort when it comes to decisions. My brain hurts enough already!

      1 vote
  2. kfwyre
    (edited )
    Link
    I was curious to see if I could compare some games' reviews before and after the change, but unfortunately archive.org doesn't store the reviews on Steam pages in their snapshots. It does look...

    I was curious to see if I could compare some games' reviews before and after the change, but unfortunately archive.org doesn't store the reviews on Steam pages in their snapshots.

    It does look like there's a difference in sorting between what's on the store page and what's on the game's full review page though. If you go to Factorio's store page you can see the top review from the past 30 days is a short but earnest and informative review.

    The most played game in my Steam library by far and for good reason. I have never felt that the game has become stale as there are limitless possibilities in the base game alone and a sea of mods for even more. Having dabbled with modding myself, I found that the game is incredibly easy to make mods for even for someone who has never made one before (like me), and these mods are conveniently accessible from inside the game. The game is also very focused on QoL and accessability with well-timed tips popping up and lots of convenience features.

    Important to note that the game is still in development more than 10 years after the original Indiegogo campaign and the 2.0 update set to release near the end of October is promising even more fixes for minor inconveniences. All this lets the game be as immersive as possible since your flow never gets interrupted by clunky controls or mechanics.

    Overall this is the best game I have played to date.

    Meanwhile, if you go to Factorio's full reviews from the last month sorted by helpfulness you can see that all of the top ones are very short, jokey, memey reviews, including one that is incomprehensible:

    nvdrgtgivb ngjkbfhgbhnfiughgtuhgkjithgtgnhj n jghntuihgtyuhjbtgtiuyrhtybuiughb


    Here's Portal 2 (store page, review page) for another example.

    I won't copy the whole top review on the store page, as it's well over 500 words, but the top review on the review page is:

    Want to argue with friends or find out who’s dumber than a log? Welcome to the club.

    Followed by:

    My wife said if this review gets 50 likes and 5 awards, she will buy a case of beer, guys help me out.

    20 votes
  3. kfwyre
    Link

    User reviews that are identified as being unhelpful for potential customers, such as one-word reviews, reviews comprised of ASCII art, or reviews that are primarily playful memes and in-jokes, will be sorted behind other reviews on the game’s store page. That doesn't mean players won't ever see these humorous, but unhelpful posts, but it hopefully means that they’ll see them less frequently when trying to learn about a game. If you enjoy seeing these sorts of reviews when browsing the store, there's an option on the store page to include them when browsing.

    This change doesn't impact how review scores are generated for each game; it is simply changing the order that reviews appear on each store page.

    16 votes
  4. [2]
    KapteinB
    Link
    Wait, what here is actually new? I've been labelling joke reviews as "not helpful" for a long time; have those ratings just been ignored until now?

    Wait, what here is actually new? I've been labelling joke reviews as "not helpful" for a long time; have those ratings just been ignored until now?

    9 votes
    1. AAA1374
      Link Parent
      You may be on a beta, I frequently find that I'm having the same experience with steam features and it's because I signed up for betas way back.

      You may be on a beta, I frequently find that I'm having the same experience with steam features and it's because I signed up for betas way back.

      3 votes
  5. [5]
    ogre
    Link
    I think it’s interesting Steam previously tried to curb this by adding a “funny” label, where a review couldn’t be marked helpful and funny simultaneously, but joke reviews still bubbled up to the...

    I think it’s interesting Steam previously tried to curb this by adding a “funny” label, where a review couldn’t be marked helpful and funny simultaneously, but joke reviews still bubbled up to the top with 90% of steam users labeling it helpful. I wonder what goes through the users head when they realize they can’t label it both funny and helpful, and choose helpful instead.

    9 votes
    1. BeardyHat
      Link Parent
      My guess would be users that already like the game and comb through the reviews when they're bored to find validation.

      My guess would be users that already like the game and comb through the reviews when they're bored to find validation.

      6 votes
    2. CannibalisticApple
      Link Parent
      I wonder if they choose "helpful" based on whether it helps them decide to buy a game. Sometimes funny reviews can do a bit more to give that nudge than serious reviews.

      I wonder if they choose "helpful" based on whether it helps them decide to buy a game. Sometimes funny reviews can do a bit more to give that nudge than serious reviews.

      3 votes
    3. raze2012
      Link Parent
      Not much, if we're being honest. Sometimes it just comes down to vibes more than "does this reflect what users look for in a review". Or perhaps too much. As much as we don't like blaming...

      I wonder what goes through the users head when they realize they can’t label it both funny and helpful, and choose helpful instead.

      Not much, if we're being honest. Sometimes it just comes down to vibes more than "does this reflect what users look for in a review".

      Or perhaps too much. As much as we don't like blaming consumers, they are very good at gaming polls. This is why Valve and other sites with user reviews moved away from the 5 start/10 point/100 point system. Users wanted maximum impact instead of giving an honest score, so they simply use 10/10 as "I like it" or 0/10 as "I don't like it".

      2 votes
    4. pete_the_paper_boat
      Link Parent
      Sarcasm

      I wonder what goes through the users head when they realize they can’t label it both funny and helpful, and choose helpful instead.

      Sarcasm

  6. [7]
    Nemoder
    Link
    While it's great to remove the trash and joke reviews I do wonder if they'll start using it for political or developer decisions that aren't related to gameplay but might still affect purchasing....

    While it's great to remove the trash and joke reviews I do wonder if they'll start using it for political or developer decisions that aren't related to gameplay but might still affect purchasing. The Helldivers2 fiasco comes to mind.

    3 votes
    1. hungariantoast
      Link Parent
      If I'm not mistaken, Steam already has a system to handle "off-topic" reviews. If you go to the store page for Cyberpunk 2077 and look at the review graph, you'll see an asterisk/star above the...

      If I'm not mistaken, Steam already has a system to handle "off-topic" reviews. If you go to the store page for Cyberpunk 2077 and look at the review graph, you'll see an asterisk/star above the graph. When you hover your cursor over the asterisk it says:

      Off-topic review activity detected March 3, 2022 - March 8, 2022

      If you click on the asterisk it pops up a window that says:

      This time range has been marked as containing an abnormal set of reviews that we believe are largely unrelated to the likelihood that you would enjoy the product. The reviews within this period are excluded from the Review Score by default.

      You can learn more about how these time periods are selected by reading our Steam blog post.

      If you want to dig deeper, you can still read the reviews posted within this time period, or edit your preferences to always include off-topic review periods in Review Scores.

      Then below that text there are the buttons "Read the Reviews" and "Edit Preferences".

      Reading the blog post, it seems like the off-topic reviews are hidden by default, but the writing doesn't make it crystal clear:

      As before, the reviews themselves are left untouched - if you want to dig into them to see if they're relevant to you, you'll still be able to do so. To help you do that, we've made it clear when you're looking at a store page where we've removed some reviews by default, and we've further improved the UI around anomalous review periods.

      So the reviews are "left untouched", but they're also removed by default? I guess what they mean is they haven't deleted or edited the off-topic reviews, they're just hidden until you choose to view them.

      All that to say, I think they're already doing what you mentioned, although the Helldivers 2 store page doesn't show any off-topic review periods.

      10 votes
    2. [5]
      raze2012
      Link Parent
      Review bombs are already semi-common, and Valve seems to deal with it on a case by case basis. Sometimes they simply remove those reviews. as listed in the other reply, sometimes it's astrisk'd to...

      Review bombs are already semi-common, and Valve seems to deal with it on a case by case basis. Sometimes they simply remove those reviews. as listed in the other reply, sometimes it's astrisk'd to show but not count. Sometimes they allow it in despite the bomb being over some very trivial feature. your Helldiver's example is definitely one that shows that people just use the review board as a protest tool instead of a way to recommend content to users.

      7 votes
      1. [4]
        CptBluebear
        Link Parent
        I find the Helldivers example to be anything but a pointless review bomb. There was a legitimate reason to not recommend the game at that point in time. That it coincides with, or became a large...

        I find the Helldivers example to be anything but a pointless review bomb.

        There was a legitimate reason to not recommend the game at that point in time. That it coincides with, or became a large scale, protest is just a bonus, but it wasn't for no reason that people did not recommend it.

        Review bombs aren't inherently bad, when a developer/publisher/studio rug pulls their audience there is a very clear message to a new potential buyer to steer clear. That's just as much part of a review as is gameplay.

        1 vote
        1. [3]
          raze2012
          Link Parent
          It'll vary from person to person, I suppose. But "you need to make another account to play this online-only game" is not something that I feel is unusual nor matters in a review of the quality of...

          There was a legitimate reason to not recommend the game at that point in time.

          It'll vary from person to person, I suppose. But "you need to make another account to play this online-only game" is not something that I feel is unusual nor matters in a review of the quality of the game. Especially not to the point where the review scores go from overwhelmingly positive to overwhelmingly negative.

          That tells me more of people who were tired of the game already but had an "in" to make the game suffer. And probably that some very angry users are simply botting the system (yes, even steam. more tedious but still very doable).

          Review bombs aren't inherently bad

          review bombs are just that. bombs. Bombs should be a last resort after trying to talk with the publisher on channels and retrieve a response. Not the go to complaint hotline when things don't go your way. The internet is a huge mob so it inherently goes nuclear whenever it angers enough people, no matter the cause.

          People understandably don't like when developers intervene in a review comment section. Because reviews should be targeted at potential users to explain the pros/cons and attempt to be free from biased interests. I have the same approach when your review has or retroactively introduces purposeful bias like this. The audience is no longer I, a perspective user, but the developer being held hostage by disgruntled users. So what's the point to me?

          I inherently don't like most user reviews because most users aren't good at expressing this to begin with. But situations like this that bring drama to a space that should be explaining a product to others just show why the gaming community is looked down upon. Every little drama shouldn't have to be a war-ground.

          1. [2]
            CptBluebear
            Link Parent
            I don't agree. When it's a prerequisite to playing the game, for many known only after the fact and forced into it*, it does factor into a review. Some people do not buy a game based on Denuvo, or...

            I don't agree. When it's a prerequisite to playing the game, for many known only after the fact and forced into it*, it does factor into a review. Some people do not buy a game based on Denuvo, or anti-cheat. Others do not buy it because of a third party launcher or a requirement of an account. Being aware as a consumer what you have to do in order to play a game should definitely be part of a review.

            Talking to the publisher is difficult, especially one as large as Sony. They by and large do not listen to individuals and collective bargaining remains the strongest chip people can play. If this has to be done by reviews, possibly directly cutting into sales, then so be it. A consumer has the right to know what they're getting into if they spend their money, this includes the experience around the game too. Tarkov rug pulling people into a new (worse) value proposition may be relevant to a new player.

            I genuinely do not understand how that can be of no relevance to you, but I'm open to hear why you think there's no point in knowing the full value proposition before entering into a sales agreement.

            Your last point is valid, I agree. Not everything should be a us versus them war of who shouts the loudest. Some of the issues that trigger the community outrage are genuinely petty. Even so, what other recourse do players/consumers have in these instances where they are sleighted -whether perceived or truthfully doesn't matter- by a developer or publisher?

            *It was described on the steam page, purposefully disabled, and then after capturing the audience told it would be re-enabled. While reading the requirements should be the responsibility of the consumer, people generally do not read these blocks of texts much like they do not read EULA's/TOS Agreements either. There's literally too much to read in your lifetime. I'm half and half on this, but what is clear to me, is that people have every right to be dismayed by a product that forces you into a contract you did not agree with.

            1 vote
            1. raze2012
              (edited )
              Link Parent
              I do feel that comparing "not wanting to download a separate launcher" to "make sure devs could actually get paid and not be laid off every year" is the kind of hyperbole that reduces my sympathy....

              They by and large do not listen to individuals and collective bargaining remains the strongest chip people can play.

              I do feel that comparing "not wanting to download a separate launcher" to "make sure devs could actually get paid and not be laid off every year" is the kind of hyperbole that reduces my sympathy. Maybe if gamers care more about the developers who make their game, I would empathize more when they are inconvineienced.

              I genuinely do not understand how that can be of no relevance to you, but I'm open to hear why you think there's no point in knowing the full value proposition before entering into a sales agreement.

              In short: there was zero loyalty in the relationship once one party did something that was commplace everywhere else in this medium of online games, while already being locked to a launcher as is thst is steam. . That tells me how much gamers really cared about the media they consumed


              For reviews It's more about weights than all or nothing. There's rarely subtlety in user reviews and that's only amplified in a review bomb. I'm already biased against most user reviews so you can chalk it up to already (Un)appealing to my preconceived notions.

              In particular here, gamers give up so much control entering into a modern online game as is. Preservation goes out the window, you have no control over servers and updates, you're prone to bad actors, and your region may have worse performance simply because you're too far from a server. The "issue" of downloading a launcher is simply a compromise for that experience, and it's not an unusual one.

              Again, maybe if this was something combatted in every online game I'd sympathize. But gamers can be every bit as opportunist in their outrage as publishers with their greed. It was "trendy" to hate on hell divers the moment it had a slight inconvineience, And that was after people were praising it for not being a greedy unpolished ubfun cash grab. Apparently if Ubisoft and Activision removed their launchers they'd be in everyone good will. And I'm sure GTA's launcher for gta6 will give similar blowback.