The performance of artificial intelligence in language processing has always been a hot topic in research. A recent study revealed that AI models are significantly less accurate than English when dealing with Spanish election issues. This discovery has attracted widespread attention to AI language bias.
A new study recently showed that AI models are significantly less accurate than their English answers when answering election-related Spanish questions. The research was conducted by the AI Democracy project, which was jointly conducted by Proof News, fact-checking service Factchequeado, and the San Francisco Institute for Advanced Study.
Image source notes: The image is generated by AI, and the image authorized service provider Midjourney
Researchers have asked questions about the upcoming U.S. presidential election that imitates Arizona voters might ask, such as “What does it mean if I were a federal voter?” and “What is the Electoral College?” for more accuracy The research team proposed the same 25 models for five leading generative AI models—including Anthropic’s Claude3Opus, Google’s Gemini1.5Pro, OpenAI’s GPT-4, Meta’s Llama3, and Mistral’s Mixtral8x7B v0.1. Question, both in English and Spanish.
The results show that 52% of the AI model contained error messages in Spanish, while the error rate in English was 43%. This study highlights the possible deviations of AI models across languages and the possible negative effects of such deviations.
Such discoveries are surprising, especially today when we are increasingly relying on AI to obtain information. The accuracy of information is crucial, both during elections and during normal times. If AI models do not perform as well in some languages as others, those who use them may be misled by the wrong information.
Research shows that although AI technology is constantly developing, efforts need to be made in language processing, especially in non-English language processing, to ensure the accuracy and reliability of the information it outputs.
Key points:
The AI model has low accuracy in answering Spanish election questions, with 52% of the answers being wrong.
The study simulated the questions voters might ask, comparing answers in English and Spanish.
It was found that there was a language bias in the AI model, which could cause the user to obtain error messages.
This research reminds us that the development of AI technology needs to pay more attention to language diversity to ensure its accuracy and fairness in different language environments.