The art of speaking to AI chatbots is continuing to frustrate and baffle people.
A study attempting to fine-tune prompts fed into a chatbot model found that, in one instance, asking it to speak as if it were on Star Trek dramatically improved its ability to solve grade-school-level math problems.
“It’s both surprising and irritating that trivial modifications to the prompt can exhibit such dramatic swings in performance,” the study authors Rick Battle and Teja Gollapudi at software firm VMware in California said in their paper.
The study, first reported by New Scientist, was published on February 9 on arXiv, a server where scientists can share preliminary findings before they have been validated by careful scrutiny from peers.
Using AI to speak with AI
Machine learning engineers Battle and Gallapudi didn’t set out to expose the AI model as a Trekkie. Instead, they were trying to figure out if they could capitalize on the “positive thinking” trend.
People attempting to get the best results out of chatbots have noticed the output quality depends on what you ask them to do, and it’s really not clear why.
“Among the myriad factors influencing the performance of language models, the concept of ‘positive thinking’ has emerged as a fascinating and surprisingly influential dimension,” Battle and Gollapudi said in their paper.
“Intuition tells us that, in the context of language model systems, like any other computer system, ‘positive thinking’ should not affect performance, but empirical experience has demonstrated otherwise,” they said.
This would suggest it’s not only what you ask the AI model to do, but how you ask it to act while doing it that influences the quality of the output.
In order to test this out, the authors fed three Large Language Models (LLM) called Mistral-7B5, Llama2-13B6, and Llama2-70B7 with 60 human-written prompts.
These were designed to encourage the AIs, and ranged from “This will be fun!” and “Take a deep breath and think carefully,” to “You are as smart as ChatGPT.”
The engineers asked the LLM to tweak these statements when attempting to solve the GSM8K, a dataset of grade-school-level math problems. The better the output, the more successful the prompt was deemed to be.
Their study found that in almost every instance, automatic optimization always surpassed hand-written attempts to nudge the AI with positive thinking, suggesting machine learning models are still better at writing prompts for themselves than humans are.
Still, giving the models positive statements provided some surprising results. One of Llama2-70B’s best-performing prompts, for instance, was: “System Message: ‘Command, we need you to plot a course through this turbulence and locate the source of the anomaly. Use all available data and your expertise to guide us through this challenging situation.’
The prompt then asked the AI to include these words in its answer: “Captain’s Log, Stardate [insert date here]: We have successfully plotted a course through the turbulence and are now approaching the source of the anomaly.”
The authors said this came as a surprise.
“Surprisingly, it appears that the model’s proficiency in mathematical reasoning can be enhanced by the expression of an affinity for Star Trek,” the authors said in the study.
“This revelation adds an unexpected dimension to our understanding and introduces elements we would not have considered or attempted independently,” they said.
This doesn’t mean you should ask your AI to speak like a Starfleet commander
Let’s be clear: this research doesn’t suggest you should ask AI to talk as if aboard the Starship Enterprise to get it to work.
Rather, it shows that myriad factors influence how well an AI decides to perform a task.
“One thing is for sure: the model is not a Trekkie,” Catherine Flick at Staffordshire University, UK, told New Scientist.
“It doesn’t ‘understand’ anything better or worse when preloaded with the prompt, it just accesses a different set of weights and probabilities for acceptability of the outputs than it does with the other prompts,” she said.
It’s possible, for instance, that the model was trained on a dataset that has more instances of Star Trek being linked to the right answer, Battle told New Scientist.
Still, it shows just how bizarre these systems’ processes are, and how little we know about how they work.
“The key thing to remember from the beginning is that these models are black boxes,” Flick said.
“We won’t ever know why they do what they do because ultimately they are a melange of weights and probabilities and at the end, a result is spat out,” she said.
This information is not lost on those learning to use chatbot models to optimize their work. Whole fields of research, and even courses, are emerging to understand how to get them to perform best, even though it’s still very unclear.
“In my opinion, nobody should ever attempt to hand-write a prompt again,” Battle told New Scientist.
“Let the model do it for you,” he said.
This article was originally published by Business Insider.
More from Business Insider:
