UX designers need to help people understand when to trust AI and when to question it.
Most of us have asked AI a question and trusted the answer only to find out later it’s either completely or partially wrong. This has given rise to a new challenge for UX designers – how to ensure that users understand the limitations of a system, what it based the answer on and how certain it is that it’s correct.
Research from Nature Machine Intelligence in 2025 showed that longer explanations aren’t enough, as these seem to increase user confidence while having no impact on accuracy. The inclusion of sources does seem to help people question responses and understand where they shouldn’t implicitly take a response as correct.
Something to try: If you are using AI as part of your customer service offering, look at the experience and ask three simple questions. Can users see where the answer came from? Can they recognise when it might be wrong? Can they easily challenge or correct it? If you’ve got it right, you will build user confidence. If you haven’t you are relying on blind trust.
SourcesFostering Appropriate Reliance on Large Language Models, CHI 2025.
What large language models know and what people think they know,
Nature Machine Intelligence.