New study finds it’s harder to turn off a robot when it’s begging for its life.

Reminds me of this section of the book "Alone Together" about testing whether people would hold a Furby upside-down while it complained about being scared:

A half century later, Baird asks under what conditions a creature is deemed alive enough for people to experience an ethical dilemma if it is distressed. She designs a Turing test not for the head but for the heart and calls it the “upside-down test.” A person is asked to invert three creatures: a Barbie doll, a Furby, and a biological gerbil. Baird’s question is simple: “How long can you hold the object upside down before your emotions make you turn it back?” Baird’s experiment assumes that a sociable robot makes new ethical demands. Why? The robot performs a psychology; many experience this as evidence of an inner life, no matter how primitive. Even those who do not think a Furby has a mind—and this, on a conscious level, includes most people—find themselves in a new place with an upside-down Furby that is whining and telling them it is scared. They feel themselves, often despite themselves, in a situation that calls for an ethical response. This usually happens at the moment when they identify with the “creature” before them, all the while knowing that it is “only a machine.”

This simultaneity of vision gives Baird the predictable results of the upside-down test. As Baird puts it, “People are willing to be carrying the Barbie around by the feet, slinging it by the hair . . . no problem.... People are not going to mess around with their gerbil.” But in the case of the Furby, people will “hold the Furby upside down for thirty seconds or so, but when it starts crying and saying it’s scared, most people feel guilty and turn it over.”

The work of neuroscientist Antonio Damasio offers insight into the origins of this guilt. Damasio describes two levels of experiencing pain. The first is a physical response to a painful stimulus. The second, a far more complex reaction, is an emotion associated with pain. This is an internal representation of the physical. When the Furby says, “Me scared,” it signals that it has crossed the line between a physical response and an emotion, the internal representation. When people hold a Furby upside down, they do something that would be painful if done to an animal. The Furby cries out—as if it were an animal. But then it says, “Me scared”—as if it were a person.

People are surprised by how upset they get in this theater of distress. And then they get upset that they are upset. They often try to reassure themselves, saying things like, “Chill, chill, it’s only a toy!” They are experiencing something new: you can feel bad about yourself for how you behave with a computer program. Adults come to the upside-down test knowing two things: the Furby is a machine and they are not torturers. By the end, with a whimpering Furby in tow, they are on new ethical terrain.

We are at the point of seeing digital objects as both creatures and machines. A series of fractured surfaces—pet, voice, machine, friend—come together to create an experience in which knowing that a Furby is a machine does not alter the feeling that you can cause it pain. Kara, a woman in her fifties, reflects on holding a moaning Furby that says it is scared. She finds it distasteful, “not because I believe that the Furby is really scared, but because I’m not willing to hear anything talk like that and respond by continuing my behavior. It feels to me that I could be hurt if I keep doing this.” For Kara, “That is not what I do.... In that moment, the Furby comes to represent how I treat creatures.”

The rest of the chapter is worth reading if this topic interests you, it goes on to talk about some pretty interesting ethical questions about designing a toy robot that looks like a human baby, and how making it respond (or not respond) to "pain" is a tough decision.

I know The Verge referenced it, but this is touched upon in The Good Place, which is an astoundingly good TV show that seems like it would never do on network television with all of the "boring" philosophy references. This is well worth the watch.

I guess this wouldn't work on AI researchers.

Maybe all those machine learning algorithms and algorithmic bias would be easier to combat if it made some kind of noise of pain when it was experiencing racist or sexist biases?