Anthropic Corporation recently announced an ambitious plan to fund the development of new benchmarks for evaluating the performance and impact of artificial intelligence models, marking an important step in the field of artificial intelligence security assessment. The program will provide funding to third-party organizations to develop tools that can effectively measure the advanced capabilities of artificial intelligence models, especially the capabilities of generative AI models. This move will not only improve the overall level of the artificial intelligence safety field, but will also provide valuable evaluation tools for the entire ecosystem and solve the current lack of high-quality safety-related evaluation tools. Anthropic’s initiative is noteworthy, but it has also prompted some discussion about its fairness and focus.
Anthropic on Monday announced the launch of a new initiative to fund the development of new benchmarks that can assess the performance and impact of artificial intelligence models, including generative models like its own Claude.
According to information posted on Anthropic’s official blog, the company will provide financial support to third-party organizations to develop tools that “effectively measure the advanced capabilities of artificial intelligence models.” Interested organizations can submit an application and evaluation will be done on a rolling basis.
Anthropic said the investment is aimed at improving the overall field of artificial intelligence security and providing valuable tools for the entire ecosystem. The company believes that developing high-quality, safety-related assessments remains challenging and demand exceeds supply.
With a focus on AI safety and social impact, the program plans to create challenging benchmarks through new tools, infrastructure, and methodologies. Anthropic specifically requested tests to assess the model's capabilities in areas such as cyberattacks, weapon modifications, manipulation or deception. In addition, the company is working on developing an "early warning system" for identifying and assessing national security and defense-related artificial intelligence risks.
Anthropic also said the new program will support research exploring the potential of artificial intelligence to aid scientific research, communicate in multiple languages, mitigate bias, and self-censorship. To achieve these goals, the company envisions building new platforms that will allow experts to develop assessments and conduct large-scale trials.
While Anthropic's move was praised, it also raised some questions. Some people believe that taking into account the company's commercial interests, the fairness of its funded projects may be affected. Additionally, some experts have expressed skepticism about some of the "catastrophic" and "deceptive" AI risks cited by Anthropic, arguing that this could distract from the more pressing issues of current AI regulation.
Anthropic hopes this initiative will help make comprehensive AI assessments the industry standard. However, it remains to be seen whether independent AI benchmark development groups will be willing to work with commercial AI vendors.
It will take time to test whether Anthropic's move can successfully promote the development of the field of artificial intelligence safety assessment and promote the establishment of more fair and comprehensive assessment standards. The long-term impact and potential limitations of its plans still require continued attention and evaluation.