24 votes

There is a strong correlation between comment quality and length

35 comments

  1. [18]
    nic
    Link
    If Tildes wishes to encourage in depth discussions, perhaps rather than sorting activity to the top, it might be better to sort a combination of comment length and votes to the top. It is easy to...

    If Tildes wishes to encourage in depth discussions, perhaps rather than sorting activity to the top, it might be better to sort a combination of comment length and votes to the top.

    It is easy to read and vote on a short title or a short comment.

    If someone takes the time to read a wall of text and vote on it, surely that counts for more.

    14 votes
    1. [8]
      chocolate
      Link Parent
      Encouraging people towards verbosity rather than conciseness seems like a backwards step.

      Encouraging people towards verbosity rather than conciseness seems like a backwards step.

      39 votes
      1. [5]
        Zeerph
        Link Parent
        That would be something like implementing a lower limit for comment length, which I don't think is what @nic is arguing for. Rather, I think that they would like to see a sorting method that...

        That would be something like implementing a lower limit for comment length, which I don't think is what @nic is arguing for. Rather, I think that they would like to see a sorting method that includes votes, tags, general length, and trust into the equation.

        12 votes
        1. [4]
          Awoo
          Link Parent
          The issue there is that when the userbase knows that you're comment length into the calculation for a quality-sort then the userbase's behaviour will change. It will deliberately try to be...

          That would be something like implementing a lower limit for comment length, which I don't think is what @nic is arguing for. Rather, I think that they would like to see a sorting method that includes votes, tags, general length, and trust into the equation.

          The issue there is that when the userbase knows that you're comment length into the calculation for a quality-sort then the userbase's behaviour will change.

          It will deliberately try to be verbose.

          This will almost certainly have unintended negative consequences. It's not very difficult to pad your messages out with things that look high effort but aren't really. I can start quoting all the people I respond to for example as a start, that would immediately pad my message length. Other users wouldn't have to read what I'm quoting and my short message would get just as much attention as it would as a short message as opposed to as a long message that gets less reads.

          The message I'm writing right now would get sorted higher than the message if I didn't quote you. The answer "Don't include quotes" doesn't work because I might not use the quote feature.

          4 votes
          1. [3]
            Zeerph
            Link Parent
            At least for "quotes", the algorithm can just not add words in a quote to the total word number. It could also check if those exact words were already said in a previous post, even if not in a...

            At least for "quotes", the algorithm can just not add words in a quote to the total word number. It could also check if those exact words were already said in a previous post, even if not in a quote, and also not include those. I'm sure people will find a way to game a similar system, so maybe we should only use it to gently persuade users in one direction or another, if we are looking for a certain kind of post, that is.

            Though, I do think any quality measures should be subtle. Perhaps we could start with a small word count, say 100 words, then multiply 1.1 by the amount of times (with the number slowly diminishing) users added a quality tag, it wouldn't have much of an effect, unless it was also tagged lots of times, but it would weight the longer post over the shorter one of equal quality tags.

            2 votes
            1. [2]
              Awoo
              Link Parent
              "At least for "quotes", the algorithm can just not add words in a quote to the total word number. It could also check if those exact words were already said in a previous post, even if not in a...

              "At least for "quotes", the algorithm can just not add words in a quote to the total word number. It could also check if those exact words were already said in a previous post, even if not in a quote, and also not include those."

              That sounds like adding a lot of processing that needs to be done on a site-wide basis.

              "Though, I do think any quality measures should be subtle. Perhaps we could start with a small word count, say 100 words, then multiply 1.1 by the amount of times (with the number slowly diminishing) users added a quality tag, it wouldn't have much of an effect, unless it was also tagged lots of times, but it would weight the longer post over the shorter one of equal quality tags."

              How should such a system approach things that break up quotes independently to respond in bitesized pieces?

              Perhaps there's a better way for @deimos to approach this that would empower the userbase - highlight potential "high effort" posts and ask the users if the algorithm is correct with a special vote option: "Is this high effort? Yes/Just Long".

              This would highlight potential high effort comments in a thread, but provide users with the tools to label them high effort or not.

              There does not need to be ANY feedback at all to a user that his post got marked as being potentially high effort or that users voted no on it. Feedback could be provided to a user that his post has been marked as high effort AFTER users have confirmed that it is high effort. This should hopefully eliminate perceived negativity by the recipient of "just long" votes because they won't know it was picked up as a possible effort-post anyway. And because it was highlighted by the system it should encourage more users to actually check it rather than skim over it because of the extra effort to read.

              3 votes
              1. Zeerph
                Link Parent
                After participating in the other thread about tags and filtering, I've come to the conclusion that it is more useful for the community as a whole, and trusted contributors especially, to mark...

                After participating in the other thread about tags and filtering, I've come to the conclusion that it is more useful for the community as a whole, and trusted contributors especially, to mark specific content, rather than any algorithmic processing. But we would first need a curated list or community agreed upon set of tags that enable such filtering. I think we could add just long (along with high effort) to a list of possible tags we can use for posts and I really like the idea of delayed feedback.

                3 votes
      2. [2]
        nic
        Link Parent
        You could be right. I'm assuming unnecessarily verbose comments wont get as many votes so would be systematically ignored. In general, I believe a twenty word comment with five hundred votes is...

        You could be right. I'm assuming unnecessarily verbose comments wont get as many votes so would be systematically ignored.

        In general, I believe a twenty word comment with five hundred votes is potentially as interesting as a five hundred word comment with twenty votes, and that the default sort could potentially consider that.

        Hacker News randomly puts new comments to the top, to avoid the problem of new comments being buried. Perhaps verbose comments could be randomly surfaced to the top for a short while along with the current top comment to perform rudimentary AB analysis.

        5 votes
        1. Crespyl
          Link Parent
          The random temporary promotion of new comments seems like a really good feature. Other sites have tried things like requiring comments to be unique (either globally or within a section or time...

          The random temporary promotion of new comments seems like a really good feature. Other sites have tried things like requiring comments to be unique (either globally or within a section or time span), or semi-opaque tools like /.'s "lameness filter" which mixes min/max length requirements with ALL-CAPS/ASCII-art detection and a few other things.

          It does seem like any given filter would make more or less sense depending on the community, the minimum acceptable level of discourse in ~talk might be a bit lower than ~writing or ~math for example, so some level of per-group customization might be desirable.

          I can see a deep rabbit hole for converting some spam filter into a "content quality estimator" based on extracting various metrics from a post (all the above; plus user trust, post vocabulary (value infrequent words), sentiment analysis, length, whitespace/formatting:length ratio, etc.). Of course this wouldn't replace or override user votes/tagging, but might be a useful tool to help surface new comments that might be worth looking at.

          3 votes
    2. [4]
      Zeerph
      Link Parent
      After reading the linked article, I think I will agree that quality sorting would be a good idea, that would definitely help push the more interesting comments or topics to the top and thus...

      After reading the linked article, I think I will agree that quality sorting would be a good idea, that would definitely help push the more interesting comments or topics to the top and thus increase visibility.

      4 votes
      1. [3]
        BuckeyeSundae
        Link Parent
        The big problem is how to quantify quality. The assumption with votes (that is largely flawed) is that they reflect the desires and values of the “marketplace” of voters. We probably can do with...

        The big problem is how to quantify quality. The assumption with votes (that is largely flawed) is that they reflect the desires and values of the “marketplace” of voters.

        We probably can do with other sorting methods to let people decide how best to get to the type of content they find to be quality, but a quality-sorting method itself will be nearly impossible

        8 votes
        1. [2]
          Zeerph
          Link Parent
          Perhaps tags would be useful in that regard, when they return we might be able to tag posts as "high quality" or "low quality" and sort by the most quality tags received by trusted users (when the...

          Perhaps tags would be useful in that regard, when they return we might be able to tag posts as "high quality" or "low quality" and sort by the most quality tags received by trusted users (when the trust system is implemented).
          Would something that be doable on a per ~ basis?

          2 votes
          1. BuckeyeSundae
            (edited )
            Link Parent
            That is one option. I would see that as having similar struggles as the comment-tagging system had. Namely that people were using that system as a downvote button and there was no auditing...

            That is one option. I would see that as having similar struggles as the comment-tagging system had. Namely that people were using that system as a downvote button and there was no auditing available to make sure the tags were valid and not just people punishing people they disagreed with.

            This morning I proposed an alternative option that has an attention-related check. Simply, building a sorting mechanism out of the aggregated number of follows a thread has may stand in as a form of quality-content sorting mechanism to help people find meaningful content, and since people will only be actively following a certain number of ongoing threads lest they be spammed with notifications, that should be a meaningfully vibrant sorting method.

            1 vote
    3. [4]
      havoc
      Link Parent
      While that idea may have a place on more popular, all-encompassing forums like reddit, it makes little sense in places that are designed to focus on more constructive contributions. Not to mention...

      While that idea may have a place on more popular, all-encompassing forums like reddit, it makes little sense in places that are designed to focus on more constructive contributions.

      Not to mention that you all are reading things into that statement for what you don't have data or evidence.
      Length may be an indicator, but that's may be also all that it is. And you don't know at what length the measurement becomes indistinguishable, nor whether you can even use that correlation to qualify posts relative to each other.

      1. [3]
        nic
        Link Parent
        Tildes is currently designed to surface contributions with the most activity. Please be more constructive with your criticisms. What data would prove my theorm to you? How about the top reddit...

        it makes little sense in places that are designed to focus on more constructive contributions.

        Tildes is currently designed to surface contributions with the most activity.

        Not to mention that you all are reading things into that statement for what you don't have data or evidence.

        Please be more constructive with your criticisms.

        What data would prove my theorm to you?

        How about the top reddit comments weighted by length times score?

        1 vote
        1. [2]
          havoc
          (edited )
          Link Parent
          In the past and indirectly in the docs Deimos talked about upvote velocity and weight, so that activity itself is or would be curated. What's with the rudeness? This would only count as being more...

          Tildes is currently designed to surface contributions with the most activity.

          In the past and indirectly in the docs Deimos talked about upvote velocity and weight, so that activity itself is or would be curated.

          Please be more constructive with your criticisms.

          What's with the rudeness? This would only count as being more constructive if it were not stating the obvious.

          What data would prove my theorm to you?

          Graham just said, there is an obvious correlation between length and quality of comments. Did he mean that anecdotally as a result of his experience manually moderating the site? If so, how is he sure, his perception wasn't skewed?
          If he ran analytics, when does the comment length begin to matter, a hundred characters or a thousand? How do longer posts relate after the breaking point? How do shorter posts relate to each other? And so on.

          What I take from Graham's conclusion and similarly based on anecdotal experience agree with, it makes sense for a site to encourage its communities to reach a certain average of characters per posts. It does not make sense to base comment or submission sorting on it.

          1 vote
          1. nic
            Link Parent
            What data would prove my theorem to you? How about the top reddit comments for one month as determined by number of characters in the comment multiplied by the score of the comment?

            Not to mention that you all are reading things into that statement for what you don't have data or evidence

            What data would prove my theorem to you?

            How about the top reddit comments for one month as determined by number of characters in the comment multiplied by the score of the comment?

            1 vote
    4. Mumberthrax
      Link Parent
      Sometimes walls of text are not useful, e.g. low quality. They might say a lot, but that doesn't mean they have utility. It really depends.

      Sometimes walls of text are not useful, e.g. low quality. They might say a lot, but that doesn't mean they have utility. It really depends.

  2. [4]
    EightRoundsRapid
    Link
    I would say that I have often seen long, rambling comments that take a lot of words to say what could have been distilled down to two or three sentences. Sometimes it doesn't take five hundreds...

    I would say that I have often seen long, rambling comments that take a lot of words to say what could have been distilled down to two or three sentences.

    Sometimes it doesn't take five hundreds words to make point well.

    14 votes
    1. Zeerph
      Link Parent
      Which is why, when tags are re-implemented, maybe we could tag something with "high quality" and it would be sorted a bit closer to the top.

      Which is why, when tags are re-implemented, maybe we could tag something with "high quality" and it would be sorted a bit closer to the top.

      4 votes
    2. ajar
      Link Parent
      While I agree, I don't think it's a given. Similarly, I often see short comments that don't elaborate enough to make points clear (not to talk about short comments which don't contribute much at...

      While I agree, I don't think it's a given. Similarly, I often see short comments that don't elaborate enough to make points clear (not to talk about short comments which don't contribute much at all), but I don't think all short comments are like that.

      And I think it's true that short comments in general make for a fast-consumption atmosphere. (But long ones can cause fatigue.)

      3 votes
  3. [2]
    crius
    Link
    Quite interesting article, especially some experience and solution presented. I'd like to just leave here also an interesting idea read in one of the comment that can contribute to quality: Maybe...

    Quite interesting article, especially some experience and solution presented.

    I'd like to just leave here also an interesting idea read in one of the comment that can contribute to quality:

    Maybe karma needs to be more meaningful...   
    perhaps a reflection of who voted for you and what percentage of page views voted for you (...)
    

    It made me think how we could apply this idea to our trust/votes as we had already exchanges discussing the possibility that user with more trust have their vote weight more (talking fractions here).

    I'm a strong believer in data relative values and that means that in my eyes the current "vote" system can be good only as a raw prototype.

    If we manage to implement a vote weight based on the user trust, the "most voted" filter would become automatically the quality content filter.

    And that would be quite elegant and fitting on the site that have "promote quality" as part of its mission.

    9 votes
    1. ajar
      Link Parent
      I think the point about page (comment?) views is important. If 90% of users see only the top comments, any latecomer will only see these (since if there are many comments, I don't think most users...

      I think the point about page (comment?) views is important. If 90% of users see only the top comments, any latecomer will only see these (since if there are many comments, I don't think most users will read all of them, but I'm not sure), as they have all the odds of being the most voted.

      Also, maybe it should take into account thread votes? Right now we order by most voted top-level comments. But sometimes the most interesting discussions happen inside the threads, and might be obscured by a scarcely voted top-level comment.

      4 votes
  4. [5]
    Brian
    (edited )
    Link
    This is kind of an old article though and HN typically requires a smart person to seek it out. That might be true for HN but I don't think it's true for other forums. Twitter is good because it...

    This is kind of an old article though and HN typically requires a smart person to seek it out. That might be true for HN but I don't think it's true for other forums.

    Twitter is good because it forces idiots to be brief. Most people don't need, and probably shouldn't be allotted, 800-3000 words to express themselves.

    I read a lot of dense arguments professionally. It's irritating seeing someone write a wall of poorly thought out text thinking that's the way to move discussion or win an argument. I'm not going to spend 3 hours parsing through each sentence and explaining what's wrong for people, especially when I know they're going to produce another 1000+ word nearly nonsense rebuttal.

    6 votes
    1. [4]
      Zeerph
      Link Parent
      Surely, once tags are implemented we can mark something as nonsense or low quality or something similar, and when filtering is implemented we won't have to waste our time looking at posts tagged...

      Surely, once tags are implemented we can mark something as nonsense or low quality or something similar, and when filtering is implemented we won't have to waste our time looking at posts tagged that way.

      2 votes
      1. [3]
        Cirrus
        Link Parent
        But like before, people might use the bad tags as attacks against comments they don't like. Maybe we can try what xkcd's irc channel has done - disallow any comments that have been said before....

        But like before, people might use the bad tags as attacks against comments they don't like.

        Maybe we can try what xkcd's irc channel has done - disallow any comments that have been said before. This will eliminate a lot of noise and fluff, and will encourage people to post unique and high quality stuff.

        Original post on xkcd blag: https://blog.xkcd.com/2008/01/14/robot9000-and-xkcd-signal-attacking-noise-in-chat/

        1 vote
        1. AReluctantTilder
          Link Parent
          I think that’s an interesting take on the concept and something similar might be achievable for the sight but may require an enormous amount of resources to run

          I think that’s an interesting take on the concept and something similar might be achievable for the sight but may require an enormous amount of resources to run

          2 votes
        2. Zeerph
          Link Parent
          True, which is why I am in favour of having some of these only using positive tags giving votes diminishing returns getting the trust system going so that more trusted users can have a slightly...

          But like before, people might use the bad tags as attacks against comments they don't like.

          True, which is why I am in favour of having some of these

          • only using positive tags
          • giving votes diminishing returns
          • getting the trust system going so that more trusted users can have a slightly higher weight to their votes and/or able to negate votes that aren't useful
          • limit use of negative tags according to trust level
          2 votes
  5. Paradoxa
    Link
    Well of course longer posts would equate to higher quality posts... they require more effort. I'll end this comment early and just appreciate the irony in that.

    Well of course longer posts would equate to higher quality posts... they require more effort.

    I'll end this comment early and just appreciate the irony in that.

    2 votes
  6. [2]
    Jedi
    Link
    Haha yes 👌

    Haha yes 👌

    1 vote
    1. AReluctantTilder
      Link Parent
      And perhaps this is an example of a low effort post

      And perhaps this is an example of a low effort post

      3 votes
  7. [3]
    silva-rerum
    Link
    Hey OP thanks, this was a good read and it's been on my mind since I read it last night. Here are the points that stood out most to me: This was an interesting point because it highlights the...

    Hey OP thanks, this was a good read and it's been on my mind since I read it last night. Here are the points that stood out most to me:

    Suddenly a culture that had been more or less united was divided into haves and have-nots. I didn't realize how united the culture had been till I saw it divided. It was painful to watch... Another thing I learned from this experiment is that if you're going to distinguish between people, you better be sure you do it right. This is one problem where rapid prototyping doesn't work.

    This was an interesting point because it highlights the ramifications of how not only the best users on a forum can be treated, but also the worst ones. In the case of Tildes, the former type of user could be negatively affected if the trust system, for example, turns out to be a faulty system. The latter type of user has already been affected, as shown by the discussion going on right now about what to do about trolls on Tildes.

    The reason for the difference is that Digg is derived from Slashdot, while Reddit is derived from Delicious/popular. Digg is Slashdot with voting instead of editors, and Reddit is Delicious/popular with voting instead of bookmarking. (You can still see fossils of their origins in their graphic design.)

    I was also intrigued by this point, especially considering the post I did yesterday on the rise and fall of Delicious. I hadn't realized just how far-reaching Delicious' influence has been for online companies, so viewing services like Reddit/Tildes/Delicious as the same general concept with different functionality (voting instead of bookmarking) presents those services in an interesting light. It dovetails fittingly into the current discussions on here about the direction Tildes should go, given that it's a descendant of those aggregator+commenting/bookmarking-style websites.

    1 vote
    1. [2]
      nic
      Link Parent
      Honestly, that post is something that has been in my mind for the last five years or so, since I first saw it. It has a lot of deep insights packed into a relatively small body of text.

      Honestly, that post is something that has been in my mind for the last five years or so, since I first saw it.

      It has a lot of deep insights packed into a relatively small body of text.

      1 vote
      1. silva-rerum
        Link Parent
        Are there any other short bodies of work that have had a similar impact on you?

        Are there any other short bodies of work that have had a similar impact on you?