When AI becomes the front door to your website

The assumption usability testing is built on

Standard usability testing assumes a direct relationship between your user and your content. You design a page, you test it with users, you watch them navigate it, you improve it. The method is built on the idea that the interface is the thing being experienced.

That assumption is breaking down.

Around 60% of Google searches now end without a single click to any website - in the UK specifically, the figure is even higher, at around 69.5%. For searches where an AI Overview appears, users click through to a website in just 8% of visits, compared with 15% when no AI summary appears at all. These figures shift depending on country, sector and query type, but the direction is consistent: fewer people are reaching your website, even when your content answers their question perfectly. Which means that for a growing proportion of the people you're trying to reach, an AI-generated summary sits between them and your website - and you almost certainly haven't tested what that summary says.

This is the default experience for a huge share of informational queries - exactly the kind of queries that charities and public sector organisations are built around. “How do I apply for this service?”, “What support is available for X?”, “How does this process work?”. These are the questions your content exists to answer. And increasingly, the answers are being generated, filtered and delivered by an AI that your usability testing never accounted for.

The ghost pages problem

What DBT found when they investigated was instructive: sometimes the AI simply couldn't crawl the primary guidance page properly, so it fell back on whatever else it could reach - including an outdated document from a predecessor department that nobody had touched in years. For some queries, a member of the public's own step by step account of how they did something gets treated as more useful than an organisation's official guide, because recency or specificity can outweigh authority. Nobody fully understands how these systems weigh one source against another, and it varies a lot by query, but the practical effect tends to be the same: it crawls everything it can, weights it all as potential evidence and synthesises an answer from whatever it finds most relevant to the query rather than from whatever you intended to be the definitive answer.

In February 2024 there were 700,000 published pages on GOV.UK. Most of them were never designed to be findable - they existed in what DBT describes as the “0-view abyss.” In the old model of search, that was fine. Nobody found them. In the AI model of search, they become active misinformation risks. Every unmaintained page is a potential source for a confident, wrong answer.

Charities face a version of this too. Research into how AI represents nonprofits found that it regularly cites incorrect financial information - wrong overhead ratios, outdated income figures, fabricated programme descriptions - drawing from a mix of old annual reports, archived news coverage and data from different fiscal years. A donor asking AI whether your charity is worth supporting might get an answer based on data that’s years out of date.

This isn’t purely a hallucination problem: AI is faithfully representing content that you published and forgot about.

What a usability test can’t tell you

A usability test tells you whether users can find what they need on your website. It tells you whether the journey makes sense, whether the language is clear, whether the calls to action work. It does not tell you what an AI says about you when someone never visits your site at all.

Usability testing was designed for a world where the interface was the experience. In that world it’s still essential - but it needs to sit alongside a different kind of evaluation: one that asks what AI is saying about you, where it’s getting that information from and whether the answer it gives reflects the reality you’ve designed.

That evaluation looks different to a standard usability study. It involves auditing your content estate with AI in mind - not just asking “is this page accurate?” but “is this page the one an AI will find when someone asks this question?”. It involves testing AI responses directly: querying the questions your users are most likely to ask and reading the answers carefully. It involves treating your content as a live product with ongoing maintenance requirements rather than a static artefact that gets updated when someone notices a problem.

It's also worth thinking about liability. When Air Canada's chatbot misinformed a customer about bereavement fares, a tribunal ruled the company was responsible for what it said - regardless of whether a human had written it. The same logic will increasingly apply to any organisation whose content feeds AI-generated answers.

What good evaluation looks like

Auditing your content, mapping what's actually live against what's been forgotten, checking for gaps: these are things good content and marketing teams have always been meant to do on a rolling basis. They're also the kind of housekeeping that's easy to let slide when nobody's visiting the old pages anyway - what's changed is the cost of letting it slide. You used to be able to leave low traffic content unattended because nobody found it. Now AI finds it for you, and treats it as fact.

In the age of AI overviews, we recommend:

• Auditing what AI actually says about your organisation and services, using the questions real users ask rather than the questions you've written content to answer
• Comparing AI outputs against your authoritative content and identifying where the gaps and contradictions are
• Mapping your content estate to find the ghost pages - the low-traffic, unmaintained content that you've forgotten about but AI hasn't
• Treating AI response quality as an ongoing metric rather than a one-off check

User research generates the raw material for this - the actual questions people are asking, the journeys they’re taking, and the moments where AI answers are shaping their decisions before they ever reach you. But turning those insights into a content estate that AI represents accurately is where content strategy takes over. Getting the research and content strategy functions working together on this is, in our view, one of the more pressing things a digital team can do right now.

The shift that’s already happened

The question isn't whether your users are getting answers from AI before they reach you - they are. The question is whether those answers are accurate, and whether your organisation has designed for that reality.

User-centred design has always been built on the principle that you design for how people actually behave, not how you wish they would. This is just the latest version of that challenge. The organisations that meet it will have content that AI represents accurately, users who get the right information and services that work as intended even when nobody clicks through.

The ones that don't will keep maintaining beautiful, well-tested websites that an increasing proportion of their users will never see.