OpenAI’s Superalignment Team – Navigating the Challenges of Superintelligent AI
Amid the chaos surrounding Sam Altman’s departure and return to OpenAI, OpenAI’s Superalignment team has quietly tackled the difficult challenge of mastering AI that surpasses human intelligence. The team, which includes members such as Collin Burns, Pavel Izmailov and Leopold Aschenbrenner, presented their latest work at NeurIPS, an annual machine learning conference in New Orleans. Founded in July, the Superalignment team is led by OpenAI founder and chief scientist Ilya Sutzkever. What is their mission? Development of methods of control, regulation and control of superintelligent AI systems, superintelligent systems. Collin Burns acknowledged the possibility of matching current models with human-level intelligence, but noted the complexity of regulatory models that are smarter than us. Despite the recent upheaval within OpenAI, Sutzkever remained at the helm of the Superalignment team and signaled his commitment to the project.
But opinions are mixed in the AI research community on the alignment, with some viewing it as premature while others see it as a distraction from pressing regulatory issues. The Superalignment team is tackling the difficult task of creating suitable command and control structures for the high-performance AI systems of the future. Given the ongoing debate over the definition of “superintelligence”, the team’s approach involves using less advanced AI models (such as GPT-2) to guide more advanced models (GPT-4) in the desired direction. Colleen Burns detailed the team’s focus: “How can we make sure that the model follows the guidelines and that the model only serves the facts? Ask if the generated code is safe or risky. “That’s the kind of work we hope to accomplish with our research.”
The Superalignment team’s metaphor involves a weaker model (involving a human observer) controlling a stronger model (involving a superintelligent AI). Pavel Izmailov explained the analogy by comparing a student to a sixth grader who controls him. The weak model produces signals similar to the instructions of a sixth grader and instructs the strong model (the learner) on the proposed task.
The team believes this approach could lead to breakthroughs in solving problems such as hallucinations in AI. While the analogy isn’t perfect, OpenAI is eager to explore the community’s ideas. To encourage this, it is launching a $10 million grant program to support technical research into optimizing superintelligence, and former Google CEO Eric Schmidt is getting a portion of that money. Despite questions about Schmidt’s motivations, OpenAI insists that their research, including its code, is publicly available. “Part of our mission is to contribute not only to the safety of our samples, but also to the safety of samples from other labs and AI developed in general,” Leopold Aschenbrenner said As OpenAI ventures into the uncharted territory of superintelligence AI, the Superalignment team is committed to ensuring that the benefits of AI extend safely to all of humanity.
