Blog

Beyond Keywords: Creating Content for AI Search

Written by Helen Brooks | Nov 12, 2026, 2:58:06 AM

My team still get asked if we can "add some keywords" or (my favourite) "SEO" a piece of content that's already been written. We politely decline.

Optimising for search, whether that's traditional search engines or AI models, can't be done with a quick sprinkling of keywords. And to suggest this is the case doesn't do my specialism justice. Sure, we can force in some selected phrases but, for one thing, any writer can do that, and for another, it provides little to no real value.

This article explores why this is the case, why it matters for AI search optimisation, and why SEO experts are best placed to help brands adapt their content to changing search behaviour.

The diminishing value of keywords in an AI-search era

Google changed its algorithm from exact-match keywords to semantic, intent-based search back in 2013, when it launched its "Hummingbird" core update. This change meant that users with more specific or conversational queries started to get higher quality, more accurate results.

AI search is the natural extension of the traditional search transformation towards intent-based search. Prompts are more specific and conversational than search has ever been before.

In fact, when Rand Fiskin asked volunteers to write a prompt they might use when looking for headphones, here's what he found:

"Across 142 responses, there were barely two prompts that, if you squinted, looked similar to me at all. This taught me that AI prompting is nothing like searching Google. People don’t reduce their search intent to the fewest, most logical set of 2-5 keywords, they get creative, and weird, and highly specific."

So, why do we even use keywords?

The simplest answer is that keywords are a powerful data point. Keywords provide evidence about the questions people ask and the language they use.


Here's an example:

A user types "global b2b marketing agency" into a search engine.

The results include an essay on the history of global B2B marketing agencies, a
T-shirt shop selling shirts with the slogan “Global B2B marketing agencies are cool,” and a LinkedIn profile for a retiree who once worked at one.

These results all match the words searched, but not the likely meaning. The keyword is "global b2b marketing agency"; the intent is "I am interested in learning about top B2B marketing agencies with international capabilities".

 

Search engines are smart enough to know this. Every time they make an update to their organic algorithm, it's in pursuit of meeting the intent behind a search.

Keyword research is a process of analysis, often done at a mass scale, to determine what a brand's target audience really want to know.

SEO content without intent is like a supermarket with prominent aisle signs but no actual products. The signs say what should be there, not what is there.

The keywords are labels for questions users want answered. Or, to continue my tenuous analogy, the job of an SEO content writer is to read the labels and ensure the right produce is in the right aisles.

AI allows people to ask more complex questions

As we know, AI has changed the way we search. When the AI model is working well, a user can get a much better answer to their question because they can be more specific than keywords allow.


The user looking for a "global b2b marketing agency" has turned to an AI model. Rather than use a concise, simplified phrase, she asks:

"Provide me with a list of the top 10 B2B marketing agencies with global capabilities. Score them on results, reputation, expertise and geographic markets covered. Each agency must offer Media, SEO, PR and web development services as a minimum."

Rather than compare information from multiple pages manually, she saves herself a few hours of research and gets AI to do that for her.

In this scenario, there's no single keyword to represent the full request. She simply says exactly what she wants.

 

AI makes it easy for a user to find the exact answer they are looking for.

A study by Jenny Lee Lynch for Search Engine Land explored declining homepage clicks. In most cases, the homepage is navigational. It introduces the brand and what they offer but directs to deeper pages. Instead of answering one question in depth, it answers multiple questions quickly.

In keyword terms, branded searches are the main driver of traffic to the homepage. But Lynch highlights users' changing behaviours:

“Instead of searching for [Brand] and clicking around, they search for [Brand] pricing, [Brand] alternatives, [Brand] reviews, or [Brand] returns. Google sends them straight to the page that answers that question, often several levels deep in the site.”

AI search follows a similar pattern. In 57% of sites included in the study, the homepage was not the page that earned the most impressions in Google’s AI features. Why go via the homepage when you can go straight to the fuller answer on another page?

If your homepage is the biggest driver of AI referral traffic, it may be worth investigating whether relevant deeper pages are visible and useful to AI systems. Homepage referral data alone can’t show whether those pages are being surfaced for relevant prompts; prompt tracking and citation monitoring can help assess that.

Why can't we use AI prompt data instead of keywords?

Prompt data is difficult to quantify because AI prompts vary widely from one user to the next. Google has said that about 15% of all searches are unique i.e. they have never been searched before. But in AI almost every prompt is unique.

If an SEO knows that 3,600 people in the US search for "seo content writing services" every month, that helps them decide whether to create a webpage to meet the intent behind that search (source: Semrush, Nov 2026).

With a well-made website and enough authority (because we mustn't pretend content is the only facet of SEO), that webpage can generate valuable traffic. In short, keyword metrics provide a useful indication of what content to prioritise.

The SEO and AI Visibility team here at Gravity Global do this at a large scale as part of an Organic Landscape Analysis, grouping thousands of keywords into topics and seeing how visible a brand is for those topics compared to their search competitors.

AI is messier and harder to measure. Not only are prompts highly personalised, but AI companies don't share their prompt data with marketers in the same way that search engines share keyword data.

I won't dive into how we measure brand impact and visibility in an LLM-first world in this article but put simply: measurement for AI searches is vastly different from traditional search and requires more proxy signals and different KPIs.

Keyword research offers a proxy: by grouping related searches into topics, it shows which questions attract more interest in traditional search. It doesn’t tell us how often people ask AI tools about those topics, but it gives us a useful starting point for prioritising content.

H2: Traditional search matters for AI search experiences

Many AI systems use traditional search to help provide answers to users, either using RAG (Retrieval-Augmented Generation), where it pulls from an index of resources, or live web search. For both, ensuring your website has the right information for it to pull from is key.


A
Semrush study found that, as of February 2026, around 34.5% of ChatGPT prompts resulted in at least one web search. Research by Profound's Josh Blyskal and Jasman Singh revealed that 36.6% of Claude prompts invoked a web search.

 

Traditional search returns one set of results for a single query. In contrast, AI models may use “fan-out queries” to search across several related queries, drawing on the results to provide a more multifaceted response.


Let's say you ask an LLM:
"Provide me with a list of the top 10 B2B marketing agencies with global capabilities."

Before it provides an answer, fan-out searches might include:

  • B2B marketing agencies with global capabilities
  • Most awarded global B2B marketing agencies
  • B2B marketing agencies with offices in multiple countries
  • Reviews for B2B marketing agencies

 

Essentially, the fan-out process breaks down your query into sub-queries, so it can contextualise the answer it provides, pre-empt follow-up questions and explore potential meaning of vague questions. It is also how it handles complex, highly specific questions.


Ahrefs found that ChatGPT Deep Research made 420 searches in response to the prompt "buy red phone case".

This matters for brands because fan-out queries can reveal the related questions behind a user’s request. Understanding those questions can help identify gaps in a site’s content and where it could provide more useful information.

Expertise in search algorithms is critical for brands to succeed in search

If you're new to SEO, this may sound complicated. And it is complicated, but it isn't new. As SEOs, we've been fine-tuning our approach to intent-based search optimisation for well over a decade. Writing content optimised for users searching via AI builds on that expertise.

Organic search expertise is figuring out what content, produced in what way, will improve visibility for a website. Whether that visibility is in traditional search or AI models, it's all the same: it's about how brands turn search data into insights, awareness and action.

Learn how Gravity Global can help you publish useful answers to the questions your audiences are asking. Get in touch today.