Claude and web search: how Anthropic selects and cites its sources.

Claude searches only when it needs to: the first difference from ChatGPT

In Claude, web search is a tool the model decides to use, not a permanent mode. According to Anthropic, Claude assesses whether the question requires recent or verifiable information; if its knowledge is sufficient, it answers without searching and without citing. When it does search, it can chain several searches, each informed by the results of the previous one, before writing. A deep research mode, which multiplies these steps and produces long, sourced syntheses, is available on top for complex questions.

This selectivity has a direct consequence on what a brand can hope for. Questions of definition, general knowledge or stable method rarely trigger a search; what Claude says about them comes from its training, and is not attributed. Questions about dated facts, prices, comparisons between named offers or providers do trigger a search, and that is where your pages can appear. On our panels, the share of questions that trigger a search is lower on Claude than on ChatGPT, and much lower than on Perplexity, which searches systematically.

The decision step is the one that gets forgotten in comparisons between engines. A low citation share on Claude does not always mean your pages are being set aside; it sometimes means that no search was triggered on the questions in your panel. Telling the two cases apart is the first reading to do, by noting for each answer whether searches took place.

ClaudeBot, Claude-SearchBot, Claude-User: three agents, three roles

Anthropic documents three agents, which follow the same split as at OpenAI: training, search index, on-demand reading. Each declares itself by its name and is controlled separately in the robots.txt file, which Anthropic states it respects.

Table: scroll horizontally.

Accidental blocking takes a particular form here. ClaudeBot is known for its many visits, and many administrators have blocked it at the firewall with a rule matching the pattern "Claude". That rule also catches Claude-SearchBot and Claude-User. The site believes it has refused training; it has actually disappeared from the answers. The check is done in the server logs, agent by agent, over at least thirty days.

# Visits by the three Anthropic agents over the last 30 days, by response code
grep -E "ClaudeBot|Claude-SearchBot|Claude-User" access.log \
  | awk '{print $9, $0 ~ /Claude-SearchBot/ ? "SearchBot" : ($0 ~ /Claude-User/ ? "User" : "ClaudeBot")}' \
  | sort | uniq -c

# Expected result: 200s for SearchBot and User. 403s or 429s signal a block.

What Anthropic has documented, and what it has not said

Anthropic's transparency covers the agents, the behaviour of the search tool and the format of citations. It does not cover the index. To our knowledge, Anthropic has not described the search architecture that feeds Claude, nor the respective share of its own index and of external providers. A third-party search provider appears in the list of subprocessors published by the company, which observers noted as soon as the feature launched; the exact role of Claude-SearchBot in building the results is not detailed beyond the wording "improve the quality of search results".

What the company documents precisely concerns developers above all. The web search tool is available in Claude's programming interface, with the option of restricting searches to a list of allowed domains or excluding some. Each answer there comes with citations containing the URL, the page title and the exact passage used. This precision, which also exists in the app, is what most distinguishes Claude from the other engines, and we return to it further on.

The B2B point to remember: a growing share of Claude usage goes through enterprise assistants built on this interface, where the assistant's publisher chooses which domains are consulted. Getting onto those lists is not a matter of search optimisation but of commercial relationships and reputation; it is a separate channel, to bear in mind when a client says "we use Claude".

Accessibility requirements: what Claude must be able to read

Because Claude reads few pages, it reads them in full. The reading step by Claude-User is where we see the most silent failures: the page was found, but what comes back is empty, truncated or replaced by an error page. Four requirements cover almost every case.

The third point deserves emphasis. Claude is explicit when a page resists it: it writes that it could not access it and answers from other sources. To the user, your site then appears closed, and the competing source open. It is a direct reputational effect, visible in the conversation, which the other engines do not expose as clearly.

Claude's citation style and what it implies for your pages

Claude cites little, and it cites to the exact passage. Where ChatGPT inserts a link after a sentence and Perplexity a number, Claude attaches each claim to the exact excerpt that justifies it, which the user can consult. The answer is generally shorter, with two to five sources, and a list of sources at the end of the message.

This anchoring to the excerpt has three implications for writing.

• The passage must stand alone, word for word. What the user will read on clicking is your sentence, not a paraphrase; a sentence that begins with "this one" or "as seen above" is unusable.
• The entity's name must be in the passage. Claude cites the excerpt and the URL; if your name is not in the sentence, the brand does not appear in the answer, only in the list of sources.
• The claim must be accurate to the word. An approximate figure or an unconditional promise, reproduced verbatim, commits your credibility with a reader who checks.

On our panels, this requirement favours pages that contain short proof sentences, with a figure, a date and the named entity. These are the same properties described by the second condition in How to get cited by ChatGPT, Claude and Perplexity, pushed one step further: with Claude, the citation is a reproduction, not a rewording.

Claude versus ChatGPT: what changes for a brand

The two engines share the three-agent architecture and the principle of a search triggered by the model. They differ in how often they search, the number of sources, the form of the citation and the profile of their users. The diagram summarises these gaps as we measure them on a single panel.

Three practical differences follow. First, preparing pages for Claude is about the quality of a few sentences rather than the volume of sections. Second, measurement must note whether the search took place, otherwise citation share mixes two phenomena. Finally, the user profile differs: Claude is very present in professional and technical uses, where the question is often asked by someone who is writing, coding or analysing, and who will go and read the source. The mechanics specific to ChatGPT are detailed in ChatGPT Search: what we know about source selection; those of Perplexity, closer to Claude in its precision, in Perplexity: anatomy of an answer.

Getting ready: the checks specific to Claude

In addition to the four general conditions, here are the checks we add for this engine, in the order we run them.

1. Re-read the firewall and CDN rules looking for a "Claude" or "Anthropic" pattern applied without distinguishing between agents; replace it with three named rules.
2. Count the visits of Claude-SearchBot and Claude-User in the logs, with their response codes, over thirty days. No visits at all, on a site that other engines do cite, signals an upstream block.
3. Ask Claude to open a specific page and summarise its content. A summary that only repeats the title or the menu reveals content missing from the initial HTML.
4. Isolate, on each priority page, two or three proof sentences that contain the named entity, a figure and a date, and check that they are accurate to the word.
5. Note, in the weekly panel, whether the search was triggered, and calculate citation share on answers with a search only, in addition to the overall share.
6. List the enterprise assistants built on Claude in your sector, where your clients use them, and check whether your domains are among their allowed sources.

At Google, the logic is different: selection relies on the classic index and on an entity layer, the Knowledge Graph, which weighs more than the page itself. That is the subject of the next article, Gemini and Google: classic index, Knowledge Graph and generated answers.

What to remember