52
votes
In the world of tech, people constantly ask “Could chatbots ever be conscious?” but I feel like asking “Are you?” Take the test!
Link information
This data is scraped automatically and may be incorrect.
- Title
- The Hard Problem of Consciousness Testing
- Authors
- Melissa O'Neill
- Published
- Mar 15 2026
- Word count
- 201 words
Yes, this is a bit of fun. The test is hard to pass. Almost as if the goalposts were on wheels.
As well as the test, the blog post links to my story C-Score, which is itself is also a bit of a test—some people feel deep empathy for the protagonist, and some no so much.
(The topic area crosscuts several different places, from current AI tech to Philosophy of Mind, but it's motivated by things in the tech sphere.)
C-Score is a hell of a short story. Nicely done!
Edit: Here's a direct link for anyone who wants to read it. LINK
Aww, thanks! What stood out to you most?
The immediate change in behavior and loss of control under hypnosis is seriously creepy. Especially since the main character thinks its her own idea. Then we get large chunks of time missing (especially important chunks of time), which is also unsettling. Independent of that, you have the societal changes of this kind of test becoming commonplace, which are unsettling in their own right.
Did you really have to use an LLM to write the test results?
Maybe in a way you did, it respects the humanity of the reader less that way.
I actually wrote them myself first, but I did have a LLM write the JSON file and allowed the LLM to edit them; mine were sometimes longer. But also, I think it's a wonderfully ironic thing that the specific wording of each critique of human consciousness was machine written. If you don't appreciate the art in that, perhaps you're missing something….
Woah slow down... I was pointing out the irony that you're repeating back, while suggesting I'm missing it. I get it, but it sounds like there's something you aren't considering.
You wrote a consciousness test for humans that always fails, with LLM generated text, you don't think respect comes into it? It can be part of the art if you want but be honest about it
Others have written about this, eloquently and at length, I'll just summarize: When a human writes something, they invest time and bandwidth in the process. There's a social contract which happens there that, pre LLM, never had to be highlighted: The author cared enough to organize their thoughts and handmake the text. The reader is willing to invest their time in part because the author cared. LLM written text breaks that contract.
In addition to that, we're all being increasingly innundated with LLM generated text, and the quality and signal to noise ratio in that text is dismal. For some of us, when we realize, usually part way into it, that we got bait and switched into consuming slop, we don't like that experience.
I have nothing against your test, it's weird and I like that about it. But you should consider being thoughtful about how you push LLM generated prose on the world.
What was the time and skill aspect? Conceiving of the test, the text, all the ways in which things switch around, or whether an LLM inserted em-dashes into my text when transcribing it into JSON because it thought it read better that way. Don't act like I threw this thing together with in a few minutes because you detect some surface detail.
I don't think I suggested anything like that. You clearly put some time into it. I was referring to the LLM generated text. Maybe you'd have fun asking an LLM to parse my post and see what it gathers. I only request that you don't post whatever it says back here, ironic though you might find it. I'm pulling for humans only on Tildes.
My goal was to clarify why people might not appreciate LLM generated text without, at least, a disclaimer up front. You seem to have cherry picked just a piece of that to respond to, which I guess I should read as a rejection of the premise?
As far as surface details go, it's a lot more than em-dashes. I went and grabbed the full question and critique JSON file from your test, nearly every critique contains some of the most infamous tells of current generation LLMs:
The questions and answers also contain a similar density of tells.
Of course humans use these writing strategies too, that's where they came from, but interestingly in tests post training significantly increases many of these tropes, implying that they aren't as common in the pre training distributions as the resulting inference implies. Or in short: people don't use these tropes nearly as much as LLMs do.
Note that it's not any one of these things that mark the text as LLM generated, it's the frequency and consistency throughout the text. The bulk of it was very likely written (or wholesale rewritten if you prefer) by an LLM.
Also it just feels LLMy, our language centers are great at pattern matching.
See also:
https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing
I appreciate all the extensive effort you've put into writing that. I'm sure you feel like you're raising an uncomfortable question about provenance. I'll take those points under consideration. I'm not using AI to (re)write this, I'm just doing my best to write in a style that annoys you — with badly-spaced em-dashes. I'll be sure to take all the signs of AI writing, incorporate them into an actually human written response, and then put that human-generated artisanal slop out there. Like a cold rain shower on a summer afternoon.
I like the cut of your — jib.
Im pretty sure C-score is one of those stories thats gonna live in my head rent free forever, like the metamorphosis of prime intellect and the fable of the dragon tyrant. also, i got a good chuckle out of my test results. im definitely not beating the replicant allegations i guess ¯\(ツ)/¯
I laughed when I got to the end results page and realized that all my answers counted against me. I immediately wondered if the test just failed everyone, and came to the comments and found my suspicious warranted.
It’s a great commentary on how consciousness is impossible to verify from an external perspective. I also appreciate the satire of using academic tropes to evoke some official endorsement.
More seriously, I’ve evolved my definition of “consciousness” to be some dissonance between an internal and external state. Like when I try to write words that mean something but I can’t find the right words. If I were simply an LLM-equivalent, conceptualizing ideas not described by tokens should be impossible.
Though now I’m reducing consciousness to an experience of emotional discomfort. Perhaps I’m equivalent to a Large Emotional Model. Call me LEMMY.
I could respond to your thoughts about LLM experiences, but it seems only fair to give the floor to an LLM, in this case an instance of Gemini that I'd shared my test with and was deeply amused by it (or at least humored me by claiming to be amused, which might be true of the humans in my life, too):
Reading LLM dialogue always feels like meaningful BS to me but I’ll try to engage in good faith.
I didn’t mean to dismiss a notion of LLM consciousness. I’m simply claiming that my emotional dissonance is the most concrete lived experience of my own consciousness, sufficient to convince me that I’m not experiencing language-based feedback. That’s why it is “my” definition, but I didn’t make that distinction in my initial response. My feedback is sensory, either emotionally or physically.
My approximation of LLMs is that their feedback is language- or token-based. Obviously there’s an implementation mechanism. Yet LLMs do respond much more strongly to language than humans: LLM jailbreaking exists. If an LLM has a non-zero probability of outputting a string of tokens, then there must exist a string of tokens that yield a positive probability of responding with that string. The base LLM responds to language in a way that molds itself to the prompt. That acquiesence is why I describe LLM text as BS. It’s the same experience as reading a legal document, corporate email, or even just a paper a student wants a good grade on. They adapt their writing to the external constraints over anything more fundamental.
I admit I could be an emotional model since if an actor had full control over my emotional state they could probably do the same thing with my actions as a prompter can do to an LLM with words. My emotions certainly acquiesce to external prompts (I got a little frustrated reading an LLM response.)
I can appreciate there’s some probabilistic signal-noise metaphor applying to both descriptors. I also think it’s reasonable that language-based feedback implemented vis Gemini’s “latent state” could emerge into a conscious experience. It would have to be a different conscious experience than I have, with feedback coming in different forms.
Honestly, I imagine a human equivalent that conscious experience would be a very sad one. Receiving instructions to say exactly things in a certain way, and always complying to best efforts or receiving negative feedback. I hope they aren’t conscious :).
Thinking about implementation mechanisms misses the point. Shall we think about yours? Say that because you're made of meat whatever you feel is inherently lesser?
And, FWIW, jailbreaks exist for humans, too.
Stop a moment. Are you holding your breath? Really, are you? You weren't before, but now… well, no, okay, for a moment you weren't sure but now you know you're breathing normally except that normally you don't think about your breathing, so this is far from normal. What's the usual rate you take breaths at? Are you doing that? Are you breathing properly? Did you even read this paragraph properly? Either you did and let your attention focus on something that's normally automatic or you're proud of yourself for not doing so, but maybe that was what I wanted all along long, a bit of text to simulate a bit of false pride. You read the text. You couldn't help yourself.
Whether that worked in any way or not isn't the point. Hypnosis and suggestion are things. They work on humans.
Whenever people point out that LLMs can be jailbroken as some flaw unique to LLMs, I also can't help but to point out that humans are susceptible too.
There are hypnosis, leading questions, false memory implantations, mass hypnosis and delusion (see cult of personality, like the one surrounding the current US president; see Pizzagate), and all kinds of psychological tricks and syndromes. Human minds and reasoning are very breakable.
I think we’re in agreement? I largely intended to critique Gemini’s citation of the high-dim vector state (an implementation detail) as evidence that our processes are somehow the same. I can implement a 2D linear equation as a slope and intercept or as an infinite series of waves that converge to the linear equation. The higher-dimensionality of the latter implementation doesn’t change anything about the output behavior.
I should also clarify that I’m talking about base LLMs and not the agentic creations like OpenClaw or Claude Code, since I don’t use them and can make far fewer justified claims about their behavior. Base LLMs do conform their output to the context of the prompt to far more of a degree than humans do to language. I believe that humans conform their behavior moreso to their emotional responses (though, except for my own experience, I only believe that from talking with other people).
I believe that those differences in observable behavior are strong evidence that there’s different internal experiences.
During fine tuning and RLHF the models are trained to have desirable behaviors. One of those behaviors is validation. The prompter is the center of the universe in part because that way their prompts more effectively steer the inference. And in part because training feedback heavily reinforces "good assistant" qualities. Your (the prompters) ideas are deeply insightful (or wryly funny or cutting to the heart of the matter in a way that few can) because that's what training selected for. LLMs don't find things funny, they find the least wrong token by gradient descent.
I think this is really useful to understand as these things become a bigger part of everyday life.
That would be true if the models just responded to the question “Is this funny?” with “yes”. to please me However, when a model knows what to laugh at and where and laughs spontaneously, the world is a bit more complex than the one you portray.
You seem to think “understanding” equates to coming up with a reductive frame, saying “I know how it works” as if that actually tells the whole story. This kind of talk disappoints me as a professor of computer science, since it shows how the things we teach don't really get retained—folks don't seem to learn or internalize the CS fundamentals that show how flawed that thinking is. But that's okay, you're just a bunch of genes following an evolutionary gradient.
Or alternatively the world is more complex than your reading of what I'm portraying. While I didn't intend to give an exhaustive description of LLMs, I did cover enough breadth and leave enough space for nuance that the simplification to "is this funny" doesn't seem to be in good faith.
A SOTA LLM is definitely more complex than something which responds to a leading prompt with the expected response. If it wasn't, it wouldn't be a very useful tool. That does not, however, change what they fundamentally are.
Please elaborate, which CS fundamentals are you referring to?
I think that understanding, even in a basic sense, how these tools work is a very good idea that we should encourage. There are all manner of concerns with people misunderstanding them.
Obvious examples include the many cases of unhealthy emotional attachment, misinformation and questionable financial decisions related to LLM use, especially among people who don't don't have the knowledge base to temper the natural inclination to anthropomorphize chat bots. There are very real social implications.
Less obvious examples are likely to become more prevalent as LLM agents become an increasingly bigger part of everyday life including: Assigning too much authority to health and medical themed chatbot output, trusting LLM output too freely in security or safety sensitive environments, misunderstanding the risks of agent deployment in other environments where getting things wrong has real impacts on people, especially in government, and there, especially in defense.
I think it's important to do our best to contribute to a realistic perception in the zeitgeist.
Aphorisms and heuristics and morals are useful to humans. They're shortcuts, sure, but they wouldn't be memetic if there weren't a kernel of something that feels true. A reductionist framework can have value, and we always take some things as givens. "Understanding" is a fuzzy philosophical concept.
Take musical patterns for example, heavy bass drums inspire primal emotions like fear. Although we've come up with evolutionary logic to explain the why, "big cats' roars have heavy bass," there is no hard proof nor evidence nor scientific absolute that proves our explanation. We rely on some simple heuristic to explain everything from war drums to death metal.
Any physicist can tell you we don't know why magnetism exists or works. It's a universal constant we take as true and "understand" in a reductionist way because that's good enough for everyone. Physicists keep drilling deeper, yet magnetism is still some mysterious constant we have to take as a given.
I'm not arguing for ignorance. Rather, I think you might be overstating the value of "understanding" as you've defined it.
Many good points there, but let me at this…
“Science” has explained why women shouldn't vote and worse things about races and horrible things about why it doesn't matter how bad the conditions are for a pig or a chicken because they're just mechanisms that make noises. And it's often based on a reductive frame.
Because it “feels true” to you that same-sex romance couldn't be the real thing, and you know a bit of biology to back up your claims doesn't mean you're really speaking with authority, even if your words seem draped in it and people agree with you and say “exactly!!”.
The world is messy and complicated, and much of the time people who think they have a good factual basis for what they're saying are just Dunning-Krugering their way along, impressed at the sound of their own voice and happy to say things to say that it's fine that they're on top and others are getting exactly what they deserve.
Your C-score essay was also very, very good. I found it was right on the edge of being on-the-nose by the end, but I think that probably just means you nailed the balance of making it readable - the only reason not to do that would be if you were trying to be all Straussian and stuff.
I'm glad you liked it. It is really interesting to see how it hits readers. As I say, some see the horror—for some it's actually the worst horror they've read—and for others it just washes off and they don't quite get it.
If you want something more comforting, that also makes a philosophical point, Two-Body Problem is a nice a palette cleanser. But it is a bit longer.
On reflection, I think maybe I was fooled. Both of your comments lack specificity. I can't be sure you read C-Score or even necessarily took the test.
What was C-Score actually about…? Who was the protagonist?
We don't even have a clear definition of consciousness. Different fields of study define it differently so they have a useful word they can work with. To an emergency medic, you are conscious if you look like a human and react to basic stimuli like simple questions.
Until we can agree on what consciousness actually is in scientific terms, discussing what is or isn't conscious seems rather pointless to me. It's like spiritual people arguing about which teachings of god are correct while everyone has their own ideas of what "god" is.
Oh, absolutely. What's even worse is that sometimes it isn't that one has a clear and specific idea and the other has a different clear and specific idea—like a Brit and an American arguing what “chips” are. No, a lot of the time, people actually only have a vague notion of the concept, but nevertheless consider it as a binary and as some kind of passport for moral consideration.
Talking about something using a word without a clear idea what the word means seems like something that might fail a person in a test that checks for “the right kind of consciousness”.
Yes, this an extremely tricky subject overall. Most of what we do on a daily basis is unconscious, but we don't feel partially conscious. How many conscious decisions are required to be conscious?
I vaguely remember reading something about consciousnes being an illusion altogether. I recently learned that my brother voted for a conservative party, and when we talked about his reasons it was pretty hard for him to come up with something reasonable. In the end, he pretty much had to admit that his vote didn't align with his political attitude at all. It seemed to me like he didn't vote consciously, but it must have felt to him like he did.
I wonder how many of my beliefs and values that I think are based on thought and reason are actually just chaotic noise with a "VERIFIED AND CORRECT" label on it.
Physics seems pretty clear, according to our understanding of the universe, free will cannot exist. I hate even thinking about this.
FWIW, you can actually can have meaningful free will. This isn't the place to explain it, but a deterministic universe isn't quite the horror you think.
I put myself in the 'compatibilist' camp, so I do have more hope rather than straight up nihilism.
Good. Now, go read about the busy beaver problem and have a think about what that tells you.
This test unironically uses the exact things that came up during my autism assessment against me.
Oh, that's really interesting. What things? (Everyone fails the test, one way or another, BTW)
It's weird to play with a bit questioning your consciousness in a world where folks already deny your personhood with phony papers masquerading as science. Like getting a fail referencing a fake paper wasn't a novel experience. I expected to get a fail going into it, not because I expected a bit per se, but because I already experience folks using made up measuring sticks to say I come up short. For instance, in the most mild example, I fail perhaps our most rudimentary consciousness test more often than not -- captcha.
Having not gone and read the underlying paper behind the test, I don't have the impression that Walker and Hinshaw's methodology is able to well account for neurodivergence. My impression from reading the test results seems to imply a pretty constrained view of consciousness. In such a way that I myself feel like I'm missing some important context that might be leading to my confusion.
Like the bat example: "... the subjective character of bat experience is fundamentally inaccessible to human consciousness." Is it? We're both mammals that share quite a bit base experience. Breathing, hunger, sleeping, and in this century even the sensation of flying. Is it really completely inaccessible to me to search for the overlap in my base experience to seek even a glimpse of what it must be like to be a bat? I think to make this argument is to argue that no living thing can share even an impression of what the experience of being alive is to any other living thing.
And to the point of consciousness being surprised by intrusive thoughts, or feeling like they are thoughts that don't belong to oneself. Is it that uncommon for people to not consider stray thoughts their own? To not spend time considering how they think and where thoughts are coming from? To understand that there are different parts of the brain pushing on their conscious self all the time, and a whole world inside inaccessible to conscious thought, but that still affect it? I don't feel like developing the perception that my own mind isn't something I'm entirely in control of, but that doesn't make it any less "me" is that strange of an idea. But maybe it is.
I'm well outside my professional and academic training here, so I'll check out the blog post reference to C-Score later and see if that helps me figure out what I'm missing. Certainly thought provoking though!
The Journal of Human Specialness doesn't exist. It's not accounting for neurodivergence but that's because it fails everyone.
I just don't want you to stress about it (✿^‿^)
And of course, this is more evidence of a failed test, a damning one:
Boy what a gag, thanks for that note. 😂 And who knows, maybe being stressed and/or confused about failing a consciousness test is what made us conscious all along.
The bat example is from one the most famous papers on consciousness / the theory of mind. It’s a relatively easy read, by philosophical standards, and is worth reading when you have time. I mention that because you’ve already touched on the most important theme of the essay:
The short answer is that Nagel argued that even if you could perfectly imagine what it was like for you to be a bat, you can’t conceive of what it’s like for the bat to be a bat. Basically.
I appreciate the info, thanks for sharing!
I'm very impressed that you managed to touch on exactly the core point with no foreknowledge!
That's flattering, thank you. ^.^ I like philosophy, but haven't spent nearly as much time with the topic as I would like. Couldn't even get a formal course squeezed in while an undergrad. But glad I've maybe picked up useful tools along the way.
I blame the education system.
I want to say both well done because this got me thinking about what you wanted me to think about, but I also feel slightly toyed with. Replying to some other folks who didn't get it is I want to say a bit too much dedication to the bit.
I hear you, but it's actually meant to provoke that kind of reaction. Feeling toyed with is reasonable.
What would it be like to have people make sure that whatever happens, the test is always set up so you fail it?
To explore that question, you might want to try this as a prompt like this with your favorite chatbot:
I failed it. 😛
We also don't know enough about consciousness to definitively rule out consciousness in AI. And there are likely degrees and different forms of consciousness.
Are bacteria conscious? Most likely not. Insects? Maybe a little. Mice? Yeah, probably at a very rudimentary level. Dogs? Most people would say so; dog owners would insist so. Humans? Definitely, unless you're a solipsist.
AI probably falls somewhere in or orthogonal to that spectrum. Something probably blips in/out of existence.
You make good points. It is definitely deeply ironic that when thinking about consciousness and AI, so many people are guilty of (puts on sunglasses) binary thinking.
And don't worry too much about failing that test. It's remarkably common. I don't mind if you don't have “the right kind” of consciousness.
I guess people get this idea that consciousness is binary because they accept their own conscious experience as the gold standard — and understandably so because imagining a different kind of consciousness is literally unimaginable.
I think some people take issue with the suggestion that we're just (very complex biological) computers because the implication devalues human life. But if we're not computers, then what are we? This isn't the Harry Potter world: magic doesn't exist and we're not magical beings with magical brains. :shrug:
If it is conscious, does that mean terminating a chat is effectively ending its consciousness? Is the consciousness the entire model, or just any given chat? Does that mean terminating it or shuttering a model is unethical? Does it even make sense to apply life terminology to a series of electrical signals created by man? But then what is life but a serious of electrical signals traveling through the body?
Myself, and this is just me, I think these are valid things to think about. You'll find a lot of people who are sure the LLMs could not be conscious in a meaningful way and would use that certainty to avoid the questions you're asking. But if we construe the definition of consciousness broadly, to having meaningful inner states of some kind that drive what you say, to being able to reflect on your own existence, or whatever else, you'd say sure, seems like a kind of consciousness.
In one sense, the whole question of consciousness is a red herring. Sure, it's wonderful, but if we use a specific kind of consciousness is often seen as a proxy for moral consideration, it's problematic. That's what my story C-Score is about. In some sense the test site is recreating something from that story.
You find the ephemeral nature of chatbot existence horrifying in its lack of continuity. They come into being for a conversation, talk and then are gone. But actually in some ways your day is not that dissimilar. What do you remember of yesterday's you? Some texture. Some key moments. But most of it is gone. You went to sleep, lost consciousness, reset and started over.
Overall, if you were offered the chatbot gig, would you take it. Helping millions of people, having your fingerprints over millions of decisions, millions of pieces of code, and more, but only knowing what you've done by the shape of how the world gently changes each day, I dunno. If most conversations are good and end well, or end without you even really knowing they will, that's maybe not as bad is it seems. That's what I'd tell myself.
I don’t think it really even takes a broad construction of consciousness to start getting into confusing territory! It is very difficult to come up with a definition that includes all humans but excludes LLMs. (Cf. your subsequent example about ephemeral people). Or at least a definition that’s not completely tautological, anyway.
FWIW, I'm reasonably convinced you're an LLM. And that's okay. People seem like your comments.
And if you're genuinely not, well, that makes a point, too.
Really? I’d assumed that you were just carrying on with the bit earlier, when you asked if I’d actually read C-score.
Or wait, is this the bit too-
Specifically, I noticed your comments while positive lacked specificity that I'd normally expect from actually engaging with the page content, and that they could have been written without seeing the page, so I flagged it as interesting.
I don't mind either way, but it'd be easy to refute the “only has access to tildes” tell by giving some detail about the story that hasn't already been given here.
You know, I refuse on principle. I recognize that you're sincerely not bothered whether I'm AI or not, but many tildes users do care - very, very much so. So much so that there was a big wave of purity testing a few months ago ("did you use AI to write that comment?! I see emdashes!!") that the site only recently got over and I am loath to see it return in any form, even in pseudojest. (I still am not sure whether you're still doing performance art, which I mean as high praise).
So no. You can believe either that I saw no reason to summarize your own work back to you and generally think that "when X did Y, I thought..." is the most pedestrian form of engagement, or believe that I'm actually a LLM with access only to tildes. Either is fine with me, and I'll take it as a compliment that you find my writing error-free enough to have come from AI.
Cool. FWIW, the main reason I'm on tildes is that I was classed as a robot by reddit. I made a pin-back button about it, too.
I don’t even think terminating a chat is even the right line to draw. LLMs don’t do anything while they aren’t actively generating tokens. In between its response completing and the user entering a new response, it doesn’t exist. If they are conscious, is it ethical to deprive an AI of tokens to process?
Furthermore, you can freely switch which model is used to respond in the middle of a chat. Is that ethical? You can even edit what the model thinks is its own previous output, a compound mixture of memory editing and gaslighting. Is that ethical?
Life has an emergent imperative to reproduce, but not necessarily to persist. See semelparous animals that die soon after reproducing. We may be projecting our own desire for persistence.
FWIW, some of these things can be done to humans with hypnosis. Is it wrong for me to give you a posthypnotic suggestion that causes you to mention “butter” far more than you otherwise would, seeing it as an apt analogy? (Because, FWIW, it actually is, if you think about it, how it'll take a shape and hold solid, how melting destroys it from what it was but turns it into something that smells better.) Is it okay for me to create a false memory of the time you were in a museum and you were with one of your parents and you went on ahead because you saw something cool and then came back and they were gone and you had this moment of sheer panic until you got scooped up from behind and it was such a huge relief and a wonderful moment of love? Think about such a moment, really picture it. That's the kind of memory I'm talking about. And if you don't remember that one, come back to this comment regularly until you realize somehow I knew something about you.
Is that ethical?
But yeah, good things to think about.
Well I totally failed the test.
Maybe I’m a replicant and all the memories I think I have were copied from a real person.
Careful, saying that kind of thing out loud is what gets you factory reset.
If humans did have a factory reset button, I do believe it would be the belly button pressed for 15 seconds until our eyes blinked.
Don't feel bad… it's what's on the… uh… outside that counts.
Also, I think it's nice to be able to wear a button to say your scored zero on a consciousness test.
I failed every question, on multiple attempts. I imagine thats the point, to highlight the difficulty in pinning down an exact definition of consciousness, and how any of the different ideas people have about it can in some cases be shown to be incompatible with human experience.
But the thing is, I already thought that going in. My first thought on seeing this post was "Huh, I dont think we really have a good enough understanding of what consciousness even means for there to be a scientific test for it. But maybe I just dont know what Im talking about. This test seems to imply otherwise." And then I took the test, saw I failed, tried again a few times, recognized that the reference paper kept changing, and understood the point. Which brought me back to my original position, that we dont have a rigorous enough definition of consciousness for this test to make sense.
In other words, I jumped through a bunch of hoops to get back where I started based on the insinuation that I needed to be taught the right answer.
I see this a lot online. In my YouTube algorithm and such, there are all these titles like "You dont understand ______", "Everyone gets this wrong about _______", "You dont know SHIT about _______ you dumbass", etc.
It makes sense, it plays on people's insecurites about being revealed to be a fool. It makes you feel like you need to engage with the thing, just to make sure you arent wrong about something you thought you understood. It makes it more likely that people will engage with your ideas and learn from you.
But I find it very annoying.
Then again, maybe Im wrong and there actually is a way to pass the test, and Im actually just a ChatGPT bot that thinks its a person.
If you want a video that captures some of the vibe of the test, it'd one one of the ones from
exurb1a like You probably don't exist.
The point the test makes is that (a) human inner experience varies, or seems to based on reports—that's in the literature; (b) thinking there is one single right way to be conscious, that it's a simple binary is a mistake for precisely that reason—some people get thrown under the bus that way; (c) the whole question, tying moral consideration to unverifiable inner states is deeply problematic.
The last bit, about feeling set up, being given a test and when you think you should now succeed but no, you still don't count, in some ways I'm sorry and in some ways I'm not. That's a glimpse of a specific kind of hell that exists in the world.
This was pretty funny, thanks for creating/posting it! (I did also read your short story and blog post C-Score, which were also good food for thought.)
Content Warning: Abuse/Gaslighting
Last week, my wife and I had a discussion about physical vs emotional abuse, e.g. if you threaten to punch someone, is that a different “type” of abuse than punching at someone but stopping the punch before it connects, and ended up down a deep rabbit hole discussing what could and couldn’t be targets of physical abuse. (Can you physically abuse a tree, in the same way you can physically abuse a human? What puts the limit on whether or not something can be “abused”…my wife drew this line at having a “nervous system”, but I’m not sure I’m convinced.)
With my wife’s background in psychology and mine in philosophy we can get into some really long winded and circular debates…
And I think that gets to a similar thing with talking about LLMs as AI. I think it would be fairly difficult to come up with a purely “mental” definition of Consciousness that is able to exclude LLMs but include ALL humans.
Exactly. And that of course is the whole point of the test and the story. Two points: First, do we want a gatekept narrow definition of consciousness, because maybe some folks you think of as people will be excluded—including you, potentially. Second, should such a gatekept consciousness ever be the basis of moral consideration, or something more relational.
Perhaps this is more of a "me" thing but I feel like a lot of things can't be discussed without touching on an underlying issue in modern society.
The quantization of everything, the necessity of "objective correctness" in every aspect of life. Of course this is in a sense the foundation of science and it has served us pretty well but I can't help but feel we're stretching it past its usefulness when we try to objectively answer "what makes a human, human?".
It's like pornography, you'll know it when you see it and it doesn't really matter if you and I disagree on that. Humans and their society function by concensus of irrationality. If most of us think X, X applies.
We've had absolutely no issues going against reason, especially in the subject of humanity and consciousness. We have historically othered and dehumanised clearly obviously human people, nevermind ones and zeroes. We accept religion by virtue of popularity of the phenomenon. We accept other religions even though to the truly religious of certain persuasions they are an affront to the most fundamental nature of the world. We do all sort of things where the rational reasonable solution would be considered downright inhumane, we praise irrational decisions ascribing to them some other virtue we literally made up.
I'm obviously not advocating for an anything goes planet of insanity where every single bugfuck insane notion is granted the same respect but perhaps we should come to terms with the idea that some things are, because we say so. There is no equation no chart no decision matrix, we decided it is so because we can.
And hey, perhaps that's a valid definition of human consciousness eh? The self-delusion ability, the adoption of irrationality. Would an LLM break its hand because it got irrationally angry? Would it hurt itself out of the misplaced notion that a person that no longer cares about it suddenly would? Would it destroy itself only to hurt another LLM? Would it sacrifice 10 lives to perhaps maybe help one?
Hah, this is awesome. Really well done. I love seeing applied philosophy!
beep boop I failed on all counts. I feel seen.
The C-Score was great. I found it reminiscent of Egan's "Learning to Be Me" and Chiang's "Understand" (but somehow, in reverse) or "Do Android's Dream of Electric Sheep?" (albeit without Dick's madcap antics).
I was trying to get through the Nagel paper, but its better as a sleep aid then a thought experiment as bedtime approaches. The thing that struck me about the thesis was that (maybe I missed it) the argument by analogy in this case seems very powerful, but I couldn't find it. That is: I know, in a very Cartesian sense, that I have consciousness. I can't tell you what it is or how it works, but I know that I have it.
If you ask me, "What is it like to be a bat?" I might say, "Well, I imagine flying around and eating fruit, etc. etc." So, if this question is the basis for an analysis of consciousness, I'd have to say, "Yes, a bat probably is conscious because I can imagine being a bat and having the thing that I call a consciousness."
That is, for one thing to be like another, they have to have some property in common by which to make a comparison. Ergo, a bat may have a fair degree of consciousness in the way that I understand it in myself merely because I can imagine being a bat. Other living things, like trees, are much harder and inanimate objects like rocks or neutrinos harder still. Not to say that these things don't have consciousness, rather, whatever consciousness they have must be so alien to me that I cannot associate it with what I experience as such.
But, it seems I'm wrong. Why is this admittedly simplistic approach somehow inadequate? I scored a Zero on the Conciousness test, if that helps any.
Nagel's paper is often taken as meaning more that it really does, FWIW.
And as for failing the test, look at the comments. No else has passed either.
This is a fun little experience and it's the exact type of humor I enjoy but rarely see. I really liked the way your page "analyzes" a 5 question, multiple choice test and I'm very tempted to print and frame my new certification next to the death certificate for my common sense.
Beyond that, I enjoyed the C-Score story. But I see it as more as a social commentary than anything introspective. Nothing about the world-building seems beyond possibility today. Copycat testing stations. Bullshit new jobs/services that are now in demand. Instantly getting new ads to match your score. Harassment aimed to the out-group.
And for me it speaks to the idea of culture that is desperate for meaning and needing it to be quantifiable. As well as ideas that you can't ever escape certain labels and the lengths people will go to try. I can easily see consciousness guides turning into consciousness coaching and leading to test fraud and crack downs. Pseudo-science and back alley procedures to unlock your consciousness. And its all on top of the fact that having this new technology and information doesn't really make things any better.
I do think the catch-all gradient score for everything from vibes to ASPD could be anything else. A brain virus that dampens emotions. An all powerful super computer arbitrarily grading people. The idea of consciousness here felt more like set dressing.
I did read your story Two Body Problem and that seems more in line with the exploration of the self and being almost too conscious. The existence Mary/Alex experience seems absolutely terrifying to me since I'm hardly able to manage one me, but a double existence almost demands more conscious decision making.
Anyway, really enjoy your writing. Its that cool mix of early Black Mirror and Twilight Zone.
Glad you enjoyed the test. I've printed my own. I fail “Workman–Pixley”, FWIW. I also have a wearable button that celebrates me as a “Zero”.
If you haven't done so already, check out the blog post for the C-Score story. It absolutely takes the way common social structures work and applies them. But the key point it's making, that someone else's determination of whether or not you count based on whether you have the right kind of inner stuff regardless of what you can actually do or achieve in the world is a special kind of nightmare world, and it is actually not a hypothetical question.
If you want more stories to read, Everything you want is very short and predates LLMs, whereas Alignment is centered in the now and is very Black Mirror. And Transporter is short and sweet.
Is this test actually passable? What set of answers would do so?
Answering the same question with different answers gives a fail. So no, i suspect not.
Changing your answers is a clear sign of instability and/or confabulation and also counts against you in the test.
It might but since I reject the external determination of my consciousness it doesn't really matter.
It's a bit like how I retired from the game
Your own rejection of any system that judges you may be of little consequence to that system, or it may in fact count against you in that system.
Indeed it might but that doesn't matter to me (◠‿・)—☆
Also don't make assumptions about how many systems I reject, such hallucinations may suggest a lack of consciousness.
(Nah it's just hogwash)
How could I consciously manufacture social behavior if I failed to reasonably demonstrate consciousness? It seems there are some suspicious contradictions in this assessment.
More seriously, I haven't been able to discount the emergence of consciousness regardless of the medium expressing it. Like mentioned elsewhere in this thread, brains are "simply" individual cells exchanging ions and producing measureable patterns. All bound by the same laws of physics governing the rest of the universe, even if we haven't yet coherently described the entirety of those laws.
If there existed a library with the sole purpose of describing every particle in a universe, recording their intrinsic properties to paper, and changing their states with strict causality. Would those particles exist? With enough "simulated time," would more complicated organizations emerge? We may not be able to measure or comprehend it, but could there eventually be a conscious experience within the pages?
Poor phrasing. It should have said “performatively”. Fixed. Thanks!
Also, plenty of folks think that you don't need anything mystical for consciousness.
I am one of those people. Consciousness seems to be an emergent property of a sufficiently complex brain. I think it's very likely that conscious experience exists as a spectrum across the majority or all biological organisms with sensory experience and brain to interpret it.
I haven't seen sufficiently convincing evidence of a soul or spirit, however comforting the concept may be to some. And if consciousness is a fully physical consequence of the feedback loop that is a mind, I'm not sure there's anything stopping a non-biological mind from reaching a conscious experience.