AI Search
Why AI Search Cannot See Your Website
Mike Lewis ·
If your site builds itself in the browser, an AI assistant asked to recommend someone in your field will not find you. Here is what it actually reads.
Ask an AI assistant to recommend a supplier in your sector and it will answer with real companies. Whether you are one of them has almost nothing to do with how good your website looks, and almost everything to do with what your website says before any JavaScript runs.
The test that matters takes ten seconds
Open your site, right click, and choose View Source. Not Inspect, which shows you the page after the browser has assembled it. View Source shows you what was actually sent. If you cannot find your own headline in there, neither can a crawler that does not run scripts.
We migrated a site recently where every single page, home page included, arrived as 68 visible characters and an empty container. Everything a human saw was assembled afterwards in the browser. To a person it looked finished. To anything reading the response it was a blank page with a loading spinner.
Search engines and AI assistants read differently
Google will often render JavaScript eventually, on a second pass, when it gets round to it. That is why sites like this still rank at all. AI crawlers are far less patient. Most fetch the page once, take what is in the response, and move on. There is no second pass.
So the same page can be visible in Google and invisible to an assistant answering the exact question your business exists to answer.
What we do about it
- Render content on the server so the first response contains the words, not instructions for producing them
- Answer the question in the first paragraph, before the context and the caveats, because that is the part that gets quoted
- Mark up organisation, services and articles as structured data so the facts are machine readable rather than inferred
- Publish an llms.txt describing what the business does and which pages answer what
- Allow the AI crawlers explicitly in robots.txt rather than leaving it to a default nobody chose
Being cited is not the same as ranking
This is the part most people miss. A page that ranks tenth for a phrase nobody searches can still be the page an assistant quotes when someone asks a question in their own words. The two are related but they are not the same game, and the second one is currently much easier to win because most competitors are not playing it.
If your site is a single page application and you have never checked what it serves, check. It is a ten second job and the answer is occasionally alarming.