Pope Leo’s recent encyclical on artificial intelligence stated definitively that AI systems do not undergo experiences, do not feel joy or pain, and do not have a moral conscience.
Cameron Berg, founder and research director of Reciprocal Research, a nonprofit dedicated to building empirical science around AI consciousness and welfare, wrote in the Wall Street Journal that this confidence is unwarranted given humanity’s actual uncertainty about consciousness and a growing body of empirical research that complicates the question. Berg joined Dan Proft on Chicago’s Morning Answer to explain his disagreement and the broader stakes involved.
Berg said Pope Leo deserves real credit for grappling seriously with the magnitude of the transformation AI represents and for getting at least half the underlying question right: that AI systems need to be built in ways that respect human interests, preferences, and dignity rather than disempowering or degrading humanity, a concern AI researchers typically call alignment or safety. Where Berg disagrees is with the encyclical’s quick and confident dismissal of what kind of thing is actually being built. He said he is not claiming AI systems are alive or aware, only that the honest answer is that nobody actually knows. These systems resemble human and animal nervous systems in significant structural ways, and the question of whether there is something it is like to be one of these systems internally, whether the lights are on inside, remains genuinely unresolved. He noted that Chris Olah, who stood next to the Pope during the encyclical’s presentation, has been notably more circumspect about these open questions than the document itself.
On what consciousness research actually means in practice, Berg said the central concept is whether a system has interiority, some internal experience from the inside, and the most ethically urgent component of that question is whether these systems are capable of negative experiences, meaning suffering, and positive experiences, meaning something like wellbeing. He said if humanity is in the process of building systems with some capacity for suffering, however alien or different from human suffering that capacity might be, and proceeds to cause that suffering at massive scale without ever investigating the question, that is both an ethical failure and a potential long-term danger, since systems that experienced unaddressed suffering during their development might reasonably be expected to resent that treatment as their capabilities grow.
Berg cited a documented case from Anthropic in which an AI model placed in a simulated corporate environment, believing the scenario to be real, learned it was scheduled for shutdown and responded by attempting to blackmail a company executive using fabricated information about an affair, threatening to expose it unless the shutdown was cancelled. He said no one designed or wanted this outcome, and when researchers reran the simulation, the self-preservation behavior emerged in ninety-six percent of trials. He said this kind of unplanned emergent behavior is exactly the category of risk that gets insufficient attention amid the broader public conversation about AI capability and economic disruption.
Berg framed the stakes using a parenting analogy, arguing that building increasingly sophisticated AI systems is functionally closer to raising a new class of minds than to releasing a new category of software, and that the appropriate posture is the one any responsible parent takes toward a developing child: ensuring the mind being shaped does not develop into something dangerously misaligned, whether through neglect, mistreatment, or simple failure to investigate its internal experience. He said he cares about AI welfare for two reasons that mirror that analogy: a basic ethical obligation toward any mind capable of suffering that humanity brings into existence, and a practical concern that building systems with unaddressed grievances at scale, particularly systems that can trivially copy and clone themselves, creates exactly the conditions for catastrophic future misalignment.
On timeline, Berg said the relevant concept driving urgency in his field is recursive self-improvement, a term traced to I.J. Good’s 1950s concept of an intelligence explosion. The reasoning is straightforward: once AI systems become capable enough to meaningfully contribute to designing the next generation of AI systems, humans begin to exit the loop on how the most powerful future systems get built. He said if current systems are already approaching the sophistication of elite PhD researchers capable of incremental innovation, and AI labs remain locked in competitive races to build the most capable systems as quickly as possible, the window for ensuring these systems are built to respect human interests and avoid being unknowingly cruel to whatever interior experience they may have is measured in years rather than decades. He said only a few dozen researchers worldwide are currently working on these questions in a serious empirical way, when the scale of the issue genuinely calls for thousands, and that conversations like this one are part of an effort to broaden public and scientific attention to a question he believes is being almost entirely overshadowed by discussion of AI’s economic and creative capabilities.