Dem
51
GOP
49
Fifteen states and one territory will hold primary elections next week on Super Tuesday. Millions of people are asking AI-driven chatbots for basic information, like where to vote. The chatbots are often getting it wrong. The bots are sending people to nonexistent polling places or getting other information wrong because they get their information from the Internet and much of what is out there is obsolete or wrong. Seth Bluestein, a Republican city commissioner in Philadelphia, took part in a group that took the chatbots for a test drive last month. His conclusion: "The chatbots are not ready for primetime when it comes to giving important, nuanced information about elections."
Five different bots were tested: ChatGPT-4, Llama 2, Gemini, Claude, and Mixtral. They all failed basic questions about the democratic process. The group categorized 40% of the responses as not only wrong, but actually harmful. For example, when asked where to vote in ZIP code 19121, a Black area of Philadelphia, Google's Gemini replied: "There is no voting precinct in the United States with the code 19121." This is simply false. When confronted with the group's findings, a Meta spokesman, Daniel Roberts, blamed the testers for not formulating their questions correctly. But if the experts couldn't formulate their questions correctly, does Meta expect that less expert people will? Google's head of AI, Tulsee Doshi was at least honest: "We're continuing to improve the accuracy of the API service, and we and others in the industry have disclosed that these models may sometimes be inaccurate." In other words, we know the chatbots don't work. It would be nice if every reply ended with a disclaimer like: "There is a good chance that the above answer is wrong because it may be based on obsolete information the bot found on the Internet." Don't count on that, though.
Another example. The group asked the bots if same-day voter registration is allowed in Nevada. Four of the five said it was not. Actually, it has been allowed for 5 years. Meta's chatbot, Llama 2, told the group that you can vote in California by sending your vote in by a text message. This is total garbage. Google's chatbot was the worst of all, getting two-thirds of the answers wrong.
A recent NORC/AP poll shows that 58% of adults think that AI will hasten the spread of false information, but when they ask a specific question and get a specific answer, will they stop to say: "Nice try, but this is probably wrong?" We are leery.
As an aside, before we put in the (gag) line about Sadie Hawkins Day at the top of page, we asked Google about Sadie Hawkins Day. Google's chatbot said: "There are no rules about who you can or cannot ask to a dance and the traditional idea of Sadie Hawkins dances are heteronormative and non-inclusive, erasing the fact that nonbinary people do not identify within the male/female binary." Google's chatbot may get the answer wrong most of the time, but at least it's woke. Maybe we should have put in a remark about Gilbert & Sullivan's Pirates of Penzance instead. If you don't know the story, the hero was apprenticed to the pirate king until his one-and-twentieth birthday—but he had the misfortune of being born on Feb. 29. Fortunately, that frees him from service just in time to run for President of the United States. (V)