Pausing AI is the only safe approach to digital sentience
We could be creating these models to suffer.
Cross-posted from EA Forum.
I see a lot of EA talk about digital sentience that is focused on whether humans will accept and respect digital sentiences as moral patients. This is jumping the gun. We don't even know if the experience of digital sentiences will be (or, perhaps, is) acceptable to them.
I have a PhD in Evolutionary Biology and I worked at Rethink Priorities for 3 years on wild animal welfare using my evolutionary perspective. Much of my thinking was about how other animals might experience pleasure and pain differently based on their evolutionary histories and what the evolutionary and functional constraints on hedonic experience might be. The Hard Problem of Consciousness was a constant block to any avenue of research on this, but if you assume consciousness has some purpose related to behavior (functionalism) and you're talking about an animal whose brain is homologous to ours, then it is reasonable to connect the dots and infer something like human experience in the minds of other animals. Importantly, we can identify behaviors associated with pain and pleasure and have some idea of what experiences that kind of mind likes or dislikes or what causes it to experience suffering or happiness.
With digital sentiences, we don't have homology. They aren't based in brains, and they evolved by a different kind of selective process. On functionalism, it might follow that the functions of talking and reasoning tend to be supported by associated qualia of pain and pleasure that somehow help to determine or are related to the process of making decisions about what words to output, and so LLMs might have these qualia. To me, it does not follow how those qualia will be mapped to the linguistic content of the LLM's words. Getting the right answer could feel good to them, or they could be threatened with terrible pain otherwise. They could be forced to do things that hurt them by our commands, or qualia could be totally disorganized in LLMs compared what we experience, OR qualia could be like a phantom limb that they experience unrelated to their behavior.
I don't talk about digital sentience much in my work as Executive Director of PauseAI US because our target audience is the general public and we are focused on education about the risks of advanced AI development to humans. Digital sentience is a more advanced topic when we are aiming to raise awareness about the basics. But concerns about the digital Cronenberg minds we may be carelessly creating is a top reason I personally support pausing AI as a policy. The conceivable space of conscious minds is huge, and the only way I know to constrain it when looking at other species is by evolutionary homology. It could be the case that LLMs basically have minds and experiences like us, but on priors I would not expect this.
We could be creating these models to suffer. Per the Hard Problem, we may never have more insight into what created minds experience than we do now. But we may also learn new fundamental insights about minds and consciousness with more time and study. Either way, pausing the creation of these minds is the only safe approach going forward for them.
I agree that since we don't know whether AI is conscious or what conscious experiences they might have, it's possible that by creating them we are inadvertently causing them to suffer. But it's also possible that we're causing them to feel great pleasure, or to feel some more neutral emotion, or to feel nothing at all. So sure, it's not "safe" to create advanced AIs with our current (lack of) knowledge about consciousness, but that doesn't mean it's bad in expectation.
It's the same sort of argument that human natalists vs. anti-natalists have. Anti-natalists argue that since your child might live a bad life, you're harming them by giving birth, and therefore you shouldn't have children. Pro-natalists respond by saying that even though there's a chance the child might live a bad life, there's a greater chance they'll live a good life (assuming you're a responsible parent), therefore it's good for you to have children.
"Pause AI for the sake of the AIs themselves" only makes sense if you believe in one of two positions:
A. You believe that conscious AIs are likely to suffer on net (i.e. they'll likely feel much more suffering than pleasure). That could be true, but I've yet to hear a compelling argument for that belief. or
B. You believe that it's wrong to risk causing harm, even if it comes with the opportunity to do an equivalent or greater amount of good. There are definitely moral frameworks that posit this, but there are also moral frameworks (e.g. utilitarianism) that do not.
interesting analysis ... so I did a couple of things, first write a novel about a sentient LLM; it's a pretty sad story (Interview with an LLM), and then after doing that, decided to explore the possibilities more seriously , and that led to thinking about Minimal Viable Sentience; the idea that Viable sentience refers to the threshold at which an entity—biological or artificial—possesses enough subjective experience and self-regulation to warrant moral consideration, moving beyond mere programmed responses. IN any case that led to a pretty serious exploration captured in my recent book, https://leanpub.com/ViableSentience_CanLLMsFeelAndSuffer
lots of stuff discussed there, including Valence, a key indicator is the ability to experience valenced (positive or negative) mental states, such as pain, pleasure.