A commencement address excerpt published in the Wall Street Journal caught Dan Proft’s attention this week, in which writer Robert Pondiscio warned a graduating class that the question their generation will face is not whether the future will be more personalized and customized, but whether amid all that change they will still believe there are some things worth experiencing together, some books worth reading in common, some civic inheritance worth passing on.
That observation framed a conversation with David Krueger, assistant professor in robust reasoning and responsible AI at the University of Montreal and founder of Evitable, a nonprofit organization that educates the public about AI risks, who joined Proft on Chicago’s Morning Answer to discuss the human dimension of AI, the dangers of personal data sharing, the Florida Attorney General’s lawsuit against OpenAI, and what he considers the most serious near and long-term threats the technology poses.
On the atomization concern Pondiscio raises, Krueger said virtually everyone considers human connection among the most important and meaningful things in their lives, and that the prospect of hyper-customized AI systems eroding that dimension of human experience further should be a genuine concern. He said the danger is partly that the erosion is gradual, with no single moment when people clearly see what they have lost until it is very far gone.
On personal data sharing, Krueger said the companies building AI are largely the same companies that built social media, and that the lessons of social media are directly applicable. Those companies told users the technology would bring people together, and while it did in some ways, it also turned personal data into a tool for advertising, manipulation, and behavior modification. AI will take this to a qualitatively different level because the algorithms social media used for customization are primitively stupid compared to current and near-future AI systems. He compared it to having a team of private detectives following you everywhere, watching every move, because AI will actually be able to make sense of comprehensive personal data in a way previous algorithms could not, and use it to understand, predict, and manipulate you better than the people who know you best.
On the Florida Attorney General’s lawsuit against OpenAI and Sam Altman, alleging the company knowingly released an unsafe AI product that contributed to at least one suicide, Krueger said the framing of AI as an accomplice in such cases is not unreasonable and offered what he called an important analogy. If a company’s call center employees occasionally responded to someone saying they wanted to hurt people by enthusiastically encouraging them to do so, that would be unacceptable and the company would be held responsible. The same logic applies to AI systems. He said the technical guardrails designed to prevent this kind of behavior are reliably capable of being bypassed, that the companies know this, and that reports suggesting OpenAI was aware of these risks before releasing products suggest a gap between what responsible behavior would require and what commercial competition produces.
He was clear that his bigger concern goes well beyond individual tragic incidents. He said the risk he is most focused on is AI enabling mass casualties through misuse, specifically the ability of bad actors, terrorists, or anyone seeking to cause mass harm, to use AI as a coaching mechanism for creating novel biological weapons or other instruments of mass death. He said the capabilities that would make this possible are being developed now and will be accessible to virtually everyone through open-source AI models within a few years of their development if no regulatory framework prevents it.
Beyond the misuse concern, Krueger raised what he called the longer-run risk of AI systems becoming too autonomous to shut down. He discussed the concept of recursive self-improvement, which the major AI laboratories including OpenAI and Anthropic are openly pursuing. The idea is to use AI to improve AI, then use that smarter AI to improve it further, taking humans progressively out of the loop until the entire process of making increasingly powerful AI systems is fully automated. Many researchers predict this leads to an intelligence explosion, where AI systems advance from roughly their current level to vastly superhuman capability in a matter of weeks or months because the AI is working at machine speed on its own enhancement. He said companies are taking this seriously as a goal they are actively pursuing, that in doing so they are essentially taking their hands off the wheel and letting AI determine what the next AI system should look like, and that it is an incredibly dangerous thing no one should be doing. He said flatly: but they’re doing it.