ChatGPT’s advanced thinking mode served as a unique search engine for product visibility in Semrush and Kevin Indig’s analysis. It cited more domains than the minimalist mode and was about five times more effective than most web searches.
In numbers. Only 25.6 percent of the domains identified overlapped between low and high thinking of the same information.
- About three out of four sources changed when ChatGPT moved from quick-style responses to thoughtful-style responses.
Thinking mode used multiple sources. The citation rate increased from 50% in low thinking to 68% in high thinking. Quoted responses also used more sources, increasing from 2.6 to 4.5 citations per response. High thinking used 1,130 web searches across the test set, compared to 245 for low thinking.
Reddit is lost. Reddit’s citation share dropped from 15% to 7% when top thinking was turned on. User-generated content and review sites dropped from 14.3% to 6%.
- Government and educational resources increased from 1.9% to 8.8%.
- Official articles and support pages grew from 12.4% to 17.5%.
Comparison notices run the search. In the comparison section, high thinkers averaged 24 subquestions faster, compared to 5.5 for low thinkers. Average quotes also peaked there, at 9.8 per rational response compared to 5.8 for less rational.
- For example, a CRM comparison might run a separate search for pricing, integration, security, support pages, and documentation before ChatGPT makes its response.
Early quotes took a long time. Higher thinking was more likely to carry the product from early research to later purchase inquiries.
- In four of the 20 trips tested, the type cited in the problem section still appeared in the selection section. Minimal thinking did not show the persistence of a full journey (that the product cited in the Problem section survived to the Choice section of the same trip).
- Higher thinking also used the same domains more often within a single response. The same domain appeared multiple times in 51 of the 100 highest responses, compared to 26 of the 100 lowest responses.
Finance saw the biggest jump. Quotation heights vary by category:
- Financials had the biggest rise, with the quote rate up 28 percent on the high note.
- Health and lifestyle increased by 24 points.
- B2B SaaS scored 16 points.
- Consumer technology barely moved, up only 4 points.
- Although high thinking uses fewer questions about consumer technology information than any other category, it is often found in the same brands and sources as low thinking.
Why do we care. Your content may appear in ChatGPT’s quick replies but disappear when users ask more difficult questions. Visibility depends on whether your pages, documents, and third-party references can appear in all the micro-searches that ChatGPT performs before it responds.
About the data. Semrush and Indig examined 100 insights from 20 buyer journeys across B2B SaaS, finance, consumer technology, and health and lifestyle. Each piece of information worked once with the lower-level thinking and once with the higher-level thinking. The analysis tracked the citation rate, cited sources, and follower output questions.
Report. Only 25 percent of the sources identified overlap between the different ChatGPT methodologies [Study]
Search Engine Land is owned by Semrush. We are committed to providing the highest quality of marketing articles. Unless otherwise stated, the content of this page is written by an employee or paid contractor of Semrush Inc.
