This raises ethical questions. They train the brain by activating neurons relevant to pain/pleasure. It seems at some point, we will simulate brains of animals that people care about morally...
The process is quite complicated. Each frame of the game generates 3,335 brightness inputs and 811 R8 color inputs. These inputs are fed into the simulated fruit fly brain, which contains 25.6 million connections. These inputs hit specific neurons based on what the frame shows, and that information is used to determine which controls the fly uses. This process repeats each frame the game runs.
It even works with pain. Wormuth says that nonfatal damage targets the PPL101 dopamine receptors, which control hunger and aggression in real fruit flies.
This raises ethical questions. They train the brain by activating neurons relevant to pain/pleasure. It seems at some point, we will simulate brains of animals that people care about morally (e.g., cats, dogs, humans.)
We wouldn't think it's okay to cause a cat, dog, or a human to experience pain while being repeatedly forced to play doom. We don't know whether sim brains are conscious In The Way That Matters™, but we shouldn't be super confident that they aren't. If I'm unsure whether playing Doom in a certain way will cause a human excruciating pain, I shouldn't do it.
So my understanding of this tech is that none of these flies are conscious or even being simulated as being conscious. I think the article is overselling what this actually is. They are not...
So my understanding of this tech is that none of these flies are conscious or even being simulated as being conscious. I think the article is overselling what this actually is. They are not recreating a functioning fly brain or a fly mind; they’re using the fly’s mapped neural wiring as the structure for a computational network, then attaching Doom inputs and controls to that network through engineered interfaces.
So the “pain” claim is especially misleading. Taking damage just causes the software to feed a particular artificial input into part of that network for reinforcement. That input might be a receptor that is classified as a pain receptor to an actual fly, but it's not translating that as pain in this digital simulation, it's just an input. Frankly they could have used any input not just a pain receptor and it'd have accomplished the same thing. That is not the same thing as simulating a pain system, much less demonstrating that anything is consciously suffering.
Sorry, could you describe what a pain system is? Or how a mapped neural wiring is distinct from a functional brain ...? I was under the impression that pain is experienced in the brain, and that...
Sorry, could you describe what a pain system is? Or how a mapped neural wiring is distinct from a functional brain ...? I was under the impression that pain is experienced in the brain, and that nerves outside of the central nervous system convey information to it ... which seems analogous to inputs sent to pain receptors.
Think of it as building a computer in the exact shape and wiring of a fly's brain. The neuron map is the motherboard: every trace and every socket is in the right place. But the parts soldered...
Think of it as building a computer in the exact shape and wiring of a fly's brain. The neuron map is the motherboard: every trace and every socket is in the right place. But the parts soldered onto it aren't fly neurons. Each one is a simplified stand-in, and none of them can change or adapt. The biology that actually makes a brain run, the chemistry and hormones and neuromodulators that turn a signal into a state, isn't part of it. So the board is wired correctly, but nothing on it works the way a brain works.
Then someone plugs a keyboard into the socket labeled pain receptor. Typing triggers it. But there's no biological software there to interpret that signal as pain, and no organism for it to be pain for.
And here's the part that seals it: the actual "ouch" isn't in the fly at all. It's in the scoring rule of a small program the developer wrote, which says "bad" when the game goes wrong.
I mean ... do we know for certain that qualia are stored in the neurotransmitters, or are they an emergent property of a functioning connectome? I was under the impression that the recent flurry...
I mean ... do we know for certain that qualia are stored in the neurotransmitters, or are they an emergent property of a functioning connectome? I was under the impression that the recent flurry of interest in this area was due to the discovery that a bunch of fruit fly behaviours could be simulated with relatively simple networks.
I think your argument is that -- roughly -- you'd need to perfectly simulate biology down to the cellular level, along with neurotransmitters, hormones, neuron cell death/growth cycles (since IIRC astrocyte pruning is critical for learning, but it's been a while since my last neurobiology course), etc. in order for the simulation to "feel" something ... but I'm unaware of any definitive argumentation on the topic? Heck, people have argued that plants can feel pain. It seems difficult to say that experiences can only exist on biological computers which are absolutely identical to a mammalian brain?
(edit) I'd accept (but disagree with) an argument that only things which roughly have a vertebrate brain can think or feel, and that everything else is a dead unconscious robot, since that seems consistent. But drawing the line at simulation fidelity weirds me out for some reason.
We don't really have a working theory of consciousness, so I don't think your question is really answerable. I've played around with the fruit fly connectome myself. It's a map, not the territory...
We don't really have a working theory of consciousness, so I don't think your question is really answerable.
I've played around with the fruit fly connectome myself. It's a map, not the territory but if a map is large enough and detailed enough to be indistinguishable from the territory,what's the difference?
I think my worry here is the uncertainty regarding all of this. I don't see a strong reason to think that say cell death/growth cycles are crucial for felt experience. We have this concept of...
We don't really have a working theory of consciousness
I think my worry here is the uncertainty regarding all of this. I don't see a strong reason to think that say cell death/growth cycles are crucial for felt experience.
We have this concept of "felt experience" that has two distinct features. One, it evades description in more fundamental terms. Two, it's usually said to be sufficient for some moral worth.
If we are still very unsure about all of this when we get to, say, human brains, we should err on the side of caution. We shouldn't roll the dice on whether our Doom program causes some being the felt experience of excruciating pain.
I agree! Hence my initial questioning around a claim that the fly connectome cannot feel, let alone feel being shot by a cacodemon. I, er, perhaps should be more straightforward, and start my...
We don't really have a working theory of consciousness, so I don't think your question is really answerable.
I agree! Hence my initial questioning around a claim that the fly connectome cannot feel, let alone feel being shot by a cacodemon. I, er, perhaps should be more straightforward, and start my lines of questioning with a caveat that while I don't buy the conclusion, I want to understand the reasoning.
It’s an interesting question. If we could fully map/scan an existing human’s brain and digitize it, and then had the computational power to run that at a realtime speed, I think that would be a...
It’s an interesting question. If we could fully map/scan an existing human’s brain and digitize it, and then had the computational power to run that at a realtime speed, I think that would be a fully (digital) human brain. But what if you randomized the weights and retrained it? My guess is probably not anything resembling human-like?
Maybe at some point, but this is still pretty far. They're putting the weights of the scanned fruit-fly brain into a spiking neural network. That's not actually a very good representation of a...
It seems at some point, we will simulate brains of animals that people care about morally (e.g., cats, dogs, humans.)
Maybe at some point, but this is still pretty far. They're putting the weights of the scanned fruit-fly brain into a spiking neural network. That's not actually a very good representation of a fruit fly brain, it's just somewhat closer than a normal linear activated neural network that would be used for, say, ChatGPT, which has about as much to do with the brain as the tree datastructure has to do with the plant. A spiking neural network is like a... balanced binary tree compared to the plant. At least it can't look like a linked list?
But more or less what they're doing is initializing a neural network with a lot of parameters and more or less random weights (because there's no way the recorded weights of a fruit fly brain are in any way meaningful if plugged in directly to a spiking neural network, it's just random noise), then training it to play doom with STDP.
Random weights are actually how most neural networks are bootstrapped, so we know that works. Then it just gets trained like normal.
It's less like simulating a fruit fly's brain and more like those exercises where an artists throws a bowl of spaghetti at a canvas and then carefully turns the spaghetti stain into a beautiful portrait by painting around it.
It appears to appropriately map the functional states of the fruit fly brain. It performs fruit fly-like behavior, and training is performed by stimulating dopamine receptors. Assuming regular...
It appears to appropriately map the functional states of the fruit fly brain. It performs fruit fly-like behavior, and training is performed by stimulating dopamine receptors. Assuming regular fruit flies are conscious, are we super confident something crucial is missing from the model that's required for any kind of consciousness?
It's not enough to say there is some difference between the model brain and the fruit fly brain; that will always be true. The question is whether there's a difference that could plausibly matter for deciding that one is conscious and the other is not.
(Note: this brackets off considerations that regular fruit flies may not be conscious; in which case we can understand my point as discussing the difference between simulated brains and organic brains of a being who is uncontroversially conscious.)
It does not. A fruit fly's brain does not resemble in the least the type of spiking neural network the Brian library is used for. It's not "some difference". My analogy is that it's like comparing...
It appears to appropriately map the functional states of the fruit fly brain.
It does not. A fruit fly's brain does not resemble in the least the type of spiking neural network the Brian library is used for. It's not "some difference". My analogy is that it's like comparing an actual tree with the tree data structure. They have almost nothing to do with each other.
It performs fruit fly-like behavior, and training is performed by stimulating dopamine receptors
The authors are talking it up. It just uses STDP. Which is not how actual brains work. Maybe mildly closer than gradient descent but not by much.
The question is whether there's a difference that could plausibly matter for deciding that one is conscious and the other is not.
Perhaps, but this "fruit fly brain" is far less likely to be conscious than ChatGPT, which I also don't particularly think is conscious. So if we're to be worried about accidental consciousness, I would put more energy into the "frontier" LLMs.
Spiking Neural Networks (SNNs) are the best analogs we have to mammalian brains. This is a leading neurological theory on how brains are functionally organized. My money is on my brain being...
A fruit fly's brain does not resemble in the least the type of spiking neural network the Brian library is used for. It's not "some difference".
Spiking Neural Networks (SNNs) are the best analogs we have to mammalian brains. This is a leading neurological theory on how brains are functionally organized. My money is on my brain being functionally an SNN.
The authors are talking it up. It just uses STDP. Which is not how actual brains work.
Actually, STDP is a leading neurological theory of how mammalian brains including human brains work! [0] [1] [2] It predates the application of these neurological findings to non-biological SNNs.
Sure. They're definitely closer than a normal neural network with like RELU activations and linear neuron connections, which are so far away from human brains there's no real reason for them to be...
Spiking Neural Networks (SNNs) are the best analogs we have to mammalian brains.
Sure. They're definitely closer than a normal neural network with like RELU activations and linear neuron connections, which are so far away from human brains there's no real reason for them to be called "neural network" other than that it gets you funding faster.
They're not actually very close to how we know the brain works. Exponential activation is better, but not particularly close.
It's like a balanced binary tree vs a normal tree vs a botanical tree. A balanced binary tree is a better representation to a botanical tree, in that a unlike a generic tree data structure, it at least hast o grow in a way that doesn't turn into a line. But it's not all that close to a botanical tree, in the end.
Actually, STDP is a leading neurological theory of how mammalian brains including human brains work!
In a very loose sense. The actual mathematical operation of how STDP is applied on a SNN is very, very, very, very different than any empirical evidence we have of biological brains and how they learn.
As an “intelligent” species, we’re surprisingly ambivalent on empathy and moral/ethics. We use other beings in ways we’re overall pretty confident inflicts large amount of pain, suffering, and...
As an “intelligent” species, we’re surprisingly ambivalent on empathy and moral/ethics. We use other beings in ways we’re overall pretty confident inflicts large amount of pain, suffering, and distress and yet overall agree it’s ok to keep doing (food, fur, lab testing, vivisection, tradition).
This development is blurring a line we can’t even agree where to draw. Along with the progress on AI: at what point a synthetically/simulated brain or entity becomes analog to the thing it’s modelled after both in terms of cognition and feelings?
I agree that this is debatable, I'm highly uncertain myself. Even if they are, I think most people think fruit fly experiences and lives don't matter, at least not much. It's likely a matter of...
I agree that this is debatable, I'm highly uncertain myself. Even if they are, I think most people think fruit fly experiences and lives don't matter, at least not much.
It's likely a matter of time before we simulate rodent, dog/cat, or human brains. We should start taking seriously the ethics prior to putting sim humans through 10,000 hours of excruciating pain in Doom for an HN post.
This raises ethical questions. They train the brain by activating neurons relevant to pain/pleasure. It seems at some point, we will simulate brains of animals that people care about morally (e.g., cats, dogs, humans.)
We wouldn't think it's okay to cause a cat, dog, or a human to experience pain while being repeatedly forced to play doom. We don't know whether sim brains are conscious In The Way That Matters™, but we shouldn't be super confident that they aren't. If I'm unsure whether playing Doom in a certain way will cause a human excruciating pain, I shouldn't do it.
So my understanding of this tech is that none of these flies are conscious or even being simulated as being conscious. I think the article is overselling what this actually is. They are not recreating a functioning fly brain or a fly mind; they’re using the fly’s mapped neural wiring as the structure for a computational network, then attaching Doom inputs and controls to that network through engineered interfaces.
So the “pain” claim is especially misleading. Taking damage just causes the software to feed a particular artificial input into part of that network for reinforcement. That input might be a receptor that is classified as a pain receptor to an actual fly, but it's not translating that as pain in this digital simulation, it's just an input. Frankly they could have used any input not just a pain receptor and it'd have accomplished the same thing. That is not the same thing as simulating a pain system, much less demonstrating that anything is consciously suffering.
Sorry, could you describe what a pain system is? Or how a mapped neural wiring is distinct from a functional brain ...? I was under the impression that pain is experienced in the brain, and that nerves outside of the central nervous system convey information to it ... which seems analogous to inputs sent to pain receptors.
Think of it as building a computer in the exact shape and wiring of a fly's brain. The neuron map is the motherboard: every trace and every socket is in the right place. But the parts soldered onto it aren't fly neurons. Each one is a simplified stand-in, and none of them can change or adapt. The biology that actually makes a brain run, the chemistry and hormones and neuromodulators that turn a signal into a state, isn't part of it. So the board is wired correctly, but nothing on it works the way a brain works.
Then someone plugs a keyboard into the socket labeled pain receptor. Typing triggers it. But there's no biological software there to interpret that signal as pain, and no organism for it to be pain for.
And here's the part that seals it: the actual "ouch" isn't in the fly at all. It's in the scoring rule of a small program the developer wrote, which says "bad" when the game goes wrong.
I mean ... do we know for certain that qualia are stored in the neurotransmitters, or are they an emergent property of a functioning connectome? I was under the impression that the recent flurry of interest in this area was due to the discovery that a bunch of fruit fly behaviours could be simulated with relatively simple networks.
I think your argument is that -- roughly -- you'd need to perfectly simulate biology down to the cellular level, along with neurotransmitters, hormones, neuron cell death/growth cycles (since IIRC astrocyte pruning is critical for learning, but it's been a while since my last neurobiology course), etc. in order for the simulation to "feel" something ... but I'm unaware of any definitive argumentation on the topic? Heck, people have argued that plants can feel pain. It seems difficult to say that experiences can only exist on biological computers which are absolutely identical to a mammalian brain?
(edit) I'd accept (but disagree with) an argument that only things which roughly have a vertebrate brain can think or feel, and that everything else is a dead unconscious robot, since that seems consistent. But drawing the line at simulation fidelity weirds me out for some reason.
We don't really have a working theory of consciousness, so I don't think your question is really answerable.
I've played around with the fruit fly connectome myself. It's a map, not the territory but if a map is large enough and detailed enough to be indistinguishable from the territory,what's the difference?
I think my worry here is the uncertainty regarding all of this. I don't see a strong reason to think that say cell death/growth cycles are crucial for felt experience.
We have this concept of "felt experience" that has two distinct features. One, it evades description in more fundamental terms. Two, it's usually said to be sufficient for some moral worth.
If we are still very unsure about all of this when we get to, say, human brains, we should err on the side of caution. We shouldn't roll the dice on whether our Doom program causes some being the felt experience of excruciating pain.
I agree! Hence my initial questioning around a claim that the fly connectome cannot feel, let alone feel being shot by a cacodemon. I, er, perhaps should be more straightforward, and start my lines of questioning with a caveat that while I don't buy the conclusion, I want to understand the reasoning.
It’s an interesting question. If we could fully map/scan an existing human’s brain and digitize it, and then had the computational power to run that at a realtime speed, I think that would be a fully (digital) human brain. But what if you randomized the weights and retrained it? My guess is probably not anything resembling human-like?
You might like Permutation City by Greg Egan - it deals with basically this, in a slightly more esoteric and philosophical way
Similar to @sparkle's recommendation, this reminds me of https://qntm.org/mmacevedo
Maybe at some point, but this is still pretty far. They're putting the weights of the scanned fruit-fly brain into a spiking neural network. That's not actually a very good representation of a fruit fly brain, it's just somewhat closer than a normal linear activated neural network that would be used for, say, ChatGPT, which has about as much to do with the brain as the tree datastructure has to do with the plant. A spiking neural network is like a... balanced binary tree compared to the plant. At least it can't look like a linked list?
But more or less what they're doing is initializing a neural network with a lot of parameters and more or less random weights (because there's no way the recorded weights of a fruit fly brain are in any way meaningful if plugged in directly to a spiking neural network, it's just random noise), then training it to play doom with STDP.
Random weights are actually how most neural networks are bootstrapped, so we know that works. Then it just gets trained like normal.
It's less like simulating a fruit fly's brain and more like those exercises where an artists throws a bowl of spaghetti at a canvas and then carefully turns the spaghetti stain into a beautiful portrait by painting around it.
It appears to appropriately map the functional states of the fruit fly brain. It performs fruit fly-like behavior, and training is performed by stimulating dopamine receptors. Assuming regular fruit flies are conscious, are we super confident something crucial is missing from the model that's required for any kind of consciousness?
It's not enough to say there is some difference between the model brain and the fruit fly brain; that will always be true. The question is whether there's a difference that could plausibly matter for deciding that one is conscious and the other is not.
(Note: this brackets off considerations that regular fruit flies may not be conscious; in which case we can understand my point as discussing the difference between simulated brains and organic brains of a being who is uncontroversially conscious.)
It does not. A fruit fly's brain does not resemble in the least the type of spiking neural network the Brian library is used for. It's not "some difference". My analogy is that it's like comparing an actual tree with the tree data structure. They have almost nothing to do with each other.
The authors are talking it up. It just uses STDP. Which is not how actual brains work. Maybe mildly closer than gradient descent but not by much.
Perhaps, but this "fruit fly brain" is far less likely to be conscious than ChatGPT, which I also don't particularly think is conscious. So if we're to be worried about accidental consciousness, I would put more energy into the "frontier" LLMs.
Spiking Neural Networks (SNNs) are the best analogs we have to mammalian brains. This is a leading neurological theory on how brains are functionally organized. My money is on my brain being functionally an SNN.
Actually, STDP is a leading neurological theory of how mammalian brains including human brains work! [0] [1] [2] It predates the application of these neurological findings to non-biological SNNs.
[0] http://www.scholarpedia.org/article/Spike-timing_dependent_plasticity
[1] https://pubmed.ncbi.nlm.nih.gov/36924615/
[2] https://pmc.ncbi.nlm.nih.gov/articles/PMC2922937/
Sure. They're definitely closer than a normal neural network with like RELU activations and linear neuron connections, which are so far away from human brains there's no real reason for them to be called "neural network" other than that it gets you funding faster.
They're not actually very close to how we know the brain works. Exponential activation is better, but not particularly close.
It's like a balanced binary tree vs a normal tree vs a botanical tree. A balanced binary tree is a better representation to a botanical tree, in that a unlike a generic tree data structure, it at least hast o grow in a way that doesn't turn into a line. But it's not all that close to a botanical tree, in the end.
In a very loose sense. The actual mathematical operation of how STDP is applied on a SNN is very, very, very, very different than any empirical evidence we have of biological brains and how they learn.
As an “intelligent” species, we’re surprisingly ambivalent on empathy and moral/ethics. We use other beings in ways we’re overall pretty confident inflicts large amount of pain, suffering, and distress and yet overall agree it’s ok to keep doing (food, fur, lab testing, vivisection, tradition).
This development is blurring a line we can’t even agree where to draw. Along with the progress on AI: at what point a synthetically/simulated brain or entity becomes analog to the thing it’s modelled after both in terms of cognition and feelings?
Also I think it’s highly debatable whether an actual fruit fly is conscious in the first place.
I agree that this is debatable, I'm highly uncertain myself. Even if they are, I think most people think fruit fly experiences and lives don't matter, at least not much.
It's likely a matter of time before we simulate rodent, dog/cat, or human brains. We should start taking seriously the ethics prior to putting sim humans through 10,000 hours of excruciating pain in Doom for an HN post.