Google and the OpenAI API find every fourth page cited by ChatGPT

Tools that measure brand visibility in ChatGPT via the API or Google results may show a different set of sources than the one seen by the app user. That is the conclusion of a new study by researchers from Singapore and China.

Google organic results and OpenAI's Responses API (the interface through which companies query models from within their own software) together found 26.2 percent of the pages cited by ChatGPT in the GPT-5.5 Instant version. At the whole-site level, it was 48.7 percent. For GPT-5.3 Instant, the figures were 28.3 and 50.3 percent. These results are reported in a preprint (opens in a new tab) by researchers from the National University of Singapore and Xi'an Jiaotong University, published on arXiv on October 11. The paper has not undergone scientific peer review.

The researchers collected data from March 29 to July 29, 2026. They used 80 questions from the public Search Arena database, 40 in English and 40 in Chinese. They asked each question 20 times in GPT-5.3 Instant and GPT-5.5 Instant. They recorded data traffic in their own logged-in ChatGPT sessions, and compared citations with Google results retrieved via SerpAPI (a service for automatically retrieving search engine results) as well as with API responses with search enabled.

Google results alone identified only 6.4 percent of the pages cited by GPT-5.5 Instant. In runs with visible searches, GPT-5.3 Instant and GPT-5.5 Instant did not use the exact wording of the user's question even once, while GPT-5.2 Instant did so in 5.5 percent of runs. For questions in Chinese, GPT-5.5 Instant searched exclusively in English in 69.6 percent of runs with visible searches. The responses were also unstable: 44.5 percent of unique pages returned for a given question appeared in only one of 20 GPT-5.5 Instant runs.

The authors recommend measuring visibility both at the level of individual pages and entire websites, running the same prompt multiple times, and checking whether the tracking tool collects data via the API or directly from the ChatGPT app. The study did not include any commercial monitoring tool. It also concerns models older than GPT-5.6 and GPT-6, which OpenAI is currently rolling out, so the authors advise verifying measurement methods with every model change. The study was described, among others, by Search Engine Journal.

For companies that buy AI visibility reports, this means that, according to the authors, data from the API and Google better reflect which websites ChatGPT uses than which specific subpages it cites.