PauseAI is asking for an international treaty for a pause on frontier AI development. (And you can too!) Both the treaty and the pause could be implemented in numerous ways.
But what happens next? What do we do in Pause World?
Democratic oversight
We would find ourselves in a relative Star Trek future. The emphasis would no longer be on “how much can we build?” but how much should we build. If the treaty was overseen by a body like the United Nations, there would be democratic oversight via the democratic world powers and diplomacy with the less-than-democratic powers of Earth. Even a US-China bilateral treaty would represent an improvement in representation of Earth’s people and removal of Cold War competitive pressures to build recklessly, and form a strong base for expanding the Pause across the globe.
Open questions:
Preserving human judgement: Can world leaders consult AIs in an objective way that doesn’t compromise their judgment or loyalties? Is it safe to incorporate even narrow AI into decision making?
Research for the common good: How can we ensure that research into AI is not hijacked by special interests or by AIs themselves? Should there be a centralized research body like the Internal Atomic Energy Agency or CERN?
Enforcement and Non-proliferation: Developing new and up to date methods to verify that other entities are abiding by the Pause treaty. The Nuclear Nonproliferation Treaty included enticements to non-nuclear nations not to develop their own nuclear weapons programs, such as help with their civilian uses of nuclear energy programs. How are the benefits of AI redistributed across the world, including the nations that did not have their own AI development programs?
Scientific consensus
We have near consensus among those working on AI development now— the problem is that they all work or depend on the AI industry. Currently, academics outside of the AI companies simply cannot keep up. It takes so long to publish in academic journals that scholarship that has been through peer review is generally studying models that are a year old or more— generations in AI development time. Without private companies in proprietary control or constant, ever larger training runs, there would be the chance for independent investigators to study the process and the models, and for a legitimate scientific consensus to form.
Open questions:
Alignment: Is alignment possible? Or do we have to ensure that AI development stays below a certain line forever? What is a principled basis for that line (i.e. when do we expect certain capabilities to emerge, how will we test those hypotheses, etc.)?
Psychological concerns: What are the psychological impacts of using AI and how should we structure society around this? Can we live on the AI dole without losing meaning and purpose?
Digital sentience: Is it ethical to use AI if it may be sentient? Can we get the same capabilities without using cognizing machines?
Economic flourishing
In Pause World, we would be able to enjoy the advances of narrow intelligences. Maybe we could finally get those cancer-treating benefits we’ve heard so much about by taking the time to do real trials and implement the proper safeguards. But those advances and labor-saving would create a lot of their own problems to solve.
Open questions:
Post-labor economics: What do we do about job displacement? How do we structure society to empower individuals if they are losing the only leverage they are born with— the ability to trade their labor?
Humanity’s chosen role: What labor do we want done by humans, even if AI could do it? Should humans be the only ones in positions of responsibility? Or deep emotional labor?
Power concentration: How much wealth inequality can our society take?
But, before all that, before any actual progress is made on processing the changes that have already been introduced to our world and how to proceed safely— just imagine taking a breath 😮💨
Imagine not worrying about the firehose of AI news developments. Imagine actually knowing that serious people, overseen by other serious people in competent governance structures accountable to us, the people, were working to protect us from AI developments, instead of the hell we live in now where irresponsible industry cowboys are trying to build the biggest AI first at any cost. We can live in that world.
We can live in Pause World ⏸️. #PauseAI