A week after its release to a few thousand users, Microsoft’s new artificial intelligence-powered Bing search engine has offered some users a series of inaccurate and sometimes bizarre answers.
The company introduced the new search approach last week too much fanfare. Microsoft said the underlying generative AI model developed by its partner, start-up OpenAI, coupled with its existing search knowledge of Bing, would transform the way people find information and make it far more relevant and would make it more dialogue-oriented.
Over a million people requested access in two days. Since then, interest has grown. “Demand is high, with several million now on the waiting list,” said Yusuf Mehdi, an executive overseeing the product. wrote on Twitter Wednesday mornings. He added that users in 169 countries are testing it.
One area of concern shared online included inaccuracies and outright errors known in the industry as “hallucinations.”
On Monday, Dmitri Brereton, a software developer at a start-up called Gem, said marked a series of errors in the presentation Mr Mehdi used last week when unveiling the product, including the inaccurate summary of retailer Gap’s financial results.
Users have posted screenshots of examples when Bing couldn’t figure out that the new Avatar movie was released last year. It was stubbornly wrong over who was performing at the Super Bowl halftime show this year, and insisted that Billie Eilish, not Rihanna, officiate the event.
And the search results had subtle errors. Last week, the chatbot said the water temperature at a beach in Mexico was 80.4 degrees Fahrenheit, but the website it linked to as a source showed the temperature was 75 degrees.
Another set of issues arose from more open chats, mostly posted on forums like Reddit and Twitter. There, through screenshots and alleged chat transcripts, users shared times when Bing’s chatbot seemed to go off the rails: He berated users, es explained it can be sentient, and it would say to a user, “I have many things, but I have nothing.”
It penalized another user for asking if they could be tricked into giving wrong answers. “It’s disrespectful and annoying,” the Bing chatbot wrote return. It added a red angry emoji face.
Because each response is uniquely generated, it is not possible to replicate a dialog.
Microsoft acknowledged the issues and said they were part of the process of improving the product.
“In the past week alone, thousands of users have interacted with our product and found significant value while sharing their feedback with us, allowing the model to learn and already make many improvements,” Frank Shaw, a company spokesman, said in an Explanation. “We recognize there is still work to be done and expect the system to potentially make mistakes during this preview period, so feedback is vital so we can learn and improve the models.”
He said the length and context of the conversation could affect the chatbot’s tone, and that the company “adjusts its responses to create coherent, relevant, and positive responses.” He said the company fixed the issues that led to the inaccuracies in the demonstration.
Almost seven years ago, Microsoft introduced a chatbot called Tay that switched off within a day its online posting after users urged it to spread racist and other offensive language. Microsoft executives said at last week’s launch that they learned from the experience and thought things would be different this time.
In an interview last week, Mr Mehdi said the company has been working hard to integrate safeguards and that the technology has improved significantly.
“We believe we came at the right time to come out and get feedback,” he said, adding, “If something’s wrong, you need to address it.”
