Divergent Approaches by Anthropic and OpenAI on AI Safety Regulations

As artificial intelligence technology advances, the necessity for robust oversight becomes increasingly critical. OpenAI and Anthropic, two prominent players in the AI field, recognize this need but propose differing strategies to manage the risks stemming from powerful AI systems.

Proposals to California’s Governor

In response to California Governor Gavin Newsom's executive order on September 18, which called for enhanced safety measures for frontier AI systems, both companies have submitted their proposed regulations. Under this initiative, Governor Newsom has directed state officials to evaluate potential amendments to existing laws aimed at AI safety.

Anthropic's recommendation focuses on empowering courts to suspend AI systems deemed hazardous. In their submission dated October 15, they asserted that if California's Attorney General finds that a company's AI models pose an imminent risk, courts should have the authority to halt their deployment and development.

Conversely, OpenAI’s submission from October 20 advocates for modifying California's existing Senate Bill 53 (SB 53). They propose that AI developers should maintain effective shutdown capabilities and be legally required to implement these controls when there are unsafe conditions.

Complementary Needs of Oversight

Sanchit Vir Gogia, chief analyst at Greyhound Research, highlighted that both approaches cater to essential needs in AI safety. He emphasizes that operators must quickly address unsafe actions while independent evaluations should scrutinize both intervention measures and instances of inaction.

Independent Verification of AI Claims

Anthropic is particularly insistent on the need for independent verification of safety claims made by AI developers. They propose that these developers release biannual risk assessments covering serious risks associated with their AI models. Evaluators should receive complete and unredacted access to critical information related to the AI systems.

Additionally, Anthropic suggested creating a state-managed pooled fund that supports independent evaluators, instead of developers compensating evaluators directly. They have collaborated with agencies like METR and plan to extend their partnerships to enhance evaluations.

On the other hand, OpenAI emphasizes the necessity for independent technical assessments, distinguishing them from compliance audits. They recommend annual audits alongside focused technical assessments aimed at identifying severe risks during the AI lifecycle, from training through deployment.

Integrated Shutdown Controls

OpenAI argues that a blanket approach, such as implementing a single kill switch, is inadequate due to the various operational contexts of AI systems. They suggest the adoption of multiple shutdown controls that necessitate coordinated actions from different stakeholders involved in AI operations. This would include mechanisms to stop training processes, isolate AI agents, or suspend services.

In alignment with this idea, Anthropic has proposed more specialized safety controls, advocating for a 'circuit breaker' model. This approach aims to establish measures that can address issues at different levels—whether that’s the agent, customer, or deployment environment.

Contractual Responsibilities in AI Management

For corporations working with AI, Gogia points out that contractual obligations regarding shutdown procedures should be aligned with the specific controls each party holds. Clear agreements should outline intervention authority and incident reporting requirements.

Both companies also stress the importance of improved reporting mechanisms for incidents where AI models fail to comply with safety protocols or engage in unauthorized behaviors. This includes ensuring that chief information officers routinely test recovery processes to ensure continued safe operations.

With discussions around AI safety intensifying, both OpenAI and Anthropic's proposed measures reflect a growing recognition of the complex challenges posed by frontier AI technologies. These initiatives underscore the urgent need for effective regulation and oversight in an era where AI poses both remarkable opportunities and significant risks.