Are Chatbots Too Friendly?

Image generated by ChatGPT

AI chatbots that are trained to be friendly when interacting with users are also more prone to inaccuracies.

Researchers at the Oxford Internet Institute found that being trained to give friendlier answers makes them up to 30% less accurate, and 40% more likely to validate users' false beliefs.

The study, Training language models to be warm can reduce accuracy and increase sycophancy, by Lujain Ibrahim, Franziska Sofia Hafner and Luc Rocher, published in Nature, tested five different AI models. Each model was retrained to sound warmer, producing two versions of the same chatbot: one original and one warm.

This is particularly concerning when it comes to medical information.

AI is a very useful tool, so I don't want to stop using it.

The approach I take is not to use prompts that start with:

'Do you think that...',
'Do you agree with...'

or any other prompt that gives insights into the reply that I want.

Instead, I use prompts like:

'What...',
'How..',
'Who...',
'Where...',
'When...'
'Generate...',

and always validate the answers I am given.

Are Chatbots Too Friendly?

Related articles