13 votes

Why a social credit system is so scary

9 comments

  1. [2]
    cmccabe
    Link
    Ignoring the sales pitch for tutanota, this piece makes a good argument that we're not too far off from a social credit system like being developed in authoritarian China. All it takes is for some...

    Ignoring the sales pitch for tutanota, this piece makes a good argument that we're not too far off from a social credit system like being developed in authoritarian China. All it takes is for some entity to begin interconnecting all the individual sources of surveillance today. I would honestly be surprised if this isn't being done already.

    8 votes
  2. [5]
    Comment deleted by author
    Link
    1. [4]
      NaraVara
      Link Parent
      In a way, this is a regression to an old way of doing things. In pre-commercial civilizations stores of wealth are too transient to really assign "worth" to. So prestige is based more around an...

      There's a bigger shift going on here: in the twenty-first century social behavior will be the "new money".

      In a way, this is a regression to an old way of doing things. In pre-commercial civilizations stores of wealth are too transient to really assign "worth" to. So prestige is based more around an informal sort of "social credit" that's based on your family background, the utility of your profession, and how popular you are. This can have material impact every bit as significant as money.

      There is a story about how Queen Victoria (I think) would visit High Street shops, make a comment about how much she liked a particular thing, and just sit there drinking tea until they relent and gifted it to her. She's not paying. She's not forcing anything. It's just expected because of her informal authority as queen. Even celebrities enjoy this. They barely pay for clothes or anything, it's all sponsored. As long as they have followers it's free for them. Ditto with a lot of successful "influencers" on social media.

      So really, what the formal "social credit" we're talking about ends up doing is creates an artificial middleman to extend this older form of credit. When society gets big enough, these informal networks don't work anymore. Nobody knows who deserves credit because the social networks are too big for us to know everyone and their relationships to each other. But if we create information clearinghouses to condense those things into a "score" then we can redo some of that and approximate the favor/gift based economy that predated commercial/monetary transactions.

      Where it becomes problematic is that once we have a centralized clearinghouses, then the definitions of social credit become absolute sources of power. I don't know any system by which that doesn't become tyrannical.

      14 votes
      1. [4]
        Comment deleted by author
        Link Parent
        1. NaraVara
          Link Parent
          Sort of, except it's not a "social" credit so much as a "utility credit." Value to society doesn't enter into it and there is no "social" mechanism for determining value, it's all just the...

          Sort of, except it's not a "social" credit so much as a "utility credit." Value to society doesn't enter into it and there is no "social" mechanism for determining value, it's all just the equilibrium point of a bunch of independent, 1-to-1 trades and interactions.

          Turning how much money you have into a proxy for how valuable you are is sort of a social credit system, but it's a deliberate societal choice we made to do that. In feudal societies merchants were viewed as untrustworthy crooks and occupied positions in prestige below nobility in most cases. It only started to shift once merchant republics got rich enough to field mercenary armies that could handily defeat the less well equipped levies that barons and minor princes could muster.

          2 votes
        2. [2]
          skybrian
          (edited )
          Link Parent
          Well, both systems are about trust, but social credit is non-transferable so that's fundamentally different. Paying in cash means that the person you're paying doesn't need to trust you. You're...

          Well, both systems are about trust, but social credit is non-transferable so that's fundamentally different.

          Paying in cash means that the person you're paying doesn't need to trust you. You're essentially relying on their trust in the government or bank that issued the money. The issuer of the money needs to be trusted or it's just paper.

          Social credit might help people decide who can be trusted, but they are still trusting that person. The social credit system isn't giving you any guarantee. Saying "this person seems trustworthy" isn't like co-signing a loan.

          1 vote
          1. [2]
            Comment deleted by author
            Link Parent
            1. skybrian
              (edited )
              Link Parent
              Well, "transferable" in a financial sense would mean users could sell their social credit "points" to someone else for cash, or give them all to someone else. But once you allow that, it's no...

              Well, "transferable" in a financial sense would mean users could sell their social credit "points" to someone else for cash, or give them all to someone else. But once you allow that, it's no longer useful as a way of tracking fame or trustworthiness or whatever. Instead of being a social credit system, it's just another money substitute, like frequent flyer miles or Amazon credits.

              On the other hand, fame and reputation can sometimes be used to help someone else, for example if someone famous promotes you by resharing your posts, maybe you'll become famous too? Celebrity endorsement is big business. But it's not automatic. It's up to the audience to decide whether or not they like you too, and they can decide not to. The fans are an active party in this relationship transfer, rather just having a buyer and seller.

              Or you can always buy likes (or followers or reviews or whatever) but that's considered gaming the system.

              I guess the bigger picture here is that Internet points have different meanings depending on how you set up the rules. It's all too easy to end up with meaningless Internet points if you set it up the wrong way.

              1 vote
  3. [3]
    ubergeek
    Link
    You mean like FICO?

    You mean like FICO?

    1 vote
    1. [2]
      cmccabe
      Link Parent
      Sure, it doesn't stretch the imagination to see FICO score and more tied in through data from Equifax, Experian, TransUnion, and others -- along with data from social media, spending habits, phone...

      Sure, it doesn't stretch the imagination to see FICO score and more tied in through data from Equifax, Experian, TransUnion, and others -- along with data from social media, spending habits, phone location data, and any other behavior that can be tracked through networked apps.

      3 votes