AI optimization is currently in its earliest stages, with limited empirical research available. However, a recent study titled “GEO: Generative Engine Optimization” has begun to shed some light on this area.
The 2023 study by Aggarwal et al., published on arXiv.org, found that three specific content optimization strategies can lead to a significant 40% increase in visibility within AI-driven search results. What the writers termed “Generative Engine Optimization” provides a new framework to increase content visibility in generative AI.
While this sounds very promising, these findings require an examination of their practical application.
Table of Contents
- Generative Engine Optimization (GEO): An Overview
- Critical Observations of Generative Engine Optimization
- The Overlooked Role of SEO in AI Visibility
- Generative Engine Optimization and Ranking
- Generative Engine Optimization Categories
- Generalizability of Generative Engine Optimization
- User Engagement Metrics
- Additional Strategies for Consideration
- Multimedia Integration
- Long-Time Effectiveness of Generative Engine Optimization
- Rethinking the Name: Generative Engine Optimization (GEO)
- Proposing AI Web Optimization (AIWO)
- Final Thoughts
Generative Engine Optimization (GEO): An Overview
The study, “GEO: Generative Engine Optimization” introduces several techniques to optimize content for generative AI. These included writing in an authoritative style, citations, quotes, statistics, keyword stuffing, and others. Each strategy was designed to increase the likelihood of being featured in AI-generated responses. Notably, the three most effective generative engine optimization strategies – citations, quotes, and statistics – accounted for up to a 40% improvement.
My article ‘Generative Engine Optimization: Maximizing Visibility in AI’ explores how to test these strategies for increasing AI visibility in your content.
Critical Observations of Generative Engine Optimization
However, the Aggarwal et al. study evidenced quite a few limitations that require a discussion.
The Overlooked Role of SEO in AI Visibility
Misinterpretation of SEO’s Impact
The generative engine optimization study minimizes the foundational role of SEO in AI visibility. The authors write, “However, with generative engines becoming front-and-center in the information delivery paradigm and SEO not directly applicable to it, new techniques need to be developed.”
This suggests a limited understanding of SEO by the authors. They seem to overlook the broad scope of SEO, especially the technical aspects. Without basic SEO practices, like optimizing website crawling and indexing, AI systems might have a hard time finding content for their responses.
SEO and AI Content Discovery
AI that can access the internet for its responses needs to at root be able to find the content. SEO ensures, through robots.txt files, Meta Tags, and XML sitemaps, that content can be crawled and indexed by search engines and other entities such as AI.
Also just taking a (decidedly non-scientific) quick look at the sources for a couple of generative responses in Google’s Search Generative Experience — alpaca and Acadia National Park — and their position in search engine results pages (SERPS) shows that there could be a correlation between them. While the generative AI isn’t necessarily sourcing solely from the top 5 results of SERPS, it does appear that it may source mostly from the top 20 results.
Misconception about Keyword Stuffing
Curiously, there seems to be a misunderstanding about SEO and “keyword stuffing” in the generative engine optimization study. Aggarwal et al. write that keyword stuffing “modifies content to include more keywords from the query, as would be expected in classical SEO optimization.” However, including additional instances of a particular keyword in and of itself is not keyword stuffing.
As per Google, keyword stuffing is when “keywords appear in a list or group, unnaturally, or out of context.” The researchers could simply call the category “Keyword Optimization” to be more accurate.
For more on how SEO and Generative Engine Optimization work together to support search visibility, read “The Future of Search: Blending SEO with Generative Engine Optimization.”
Generative Engine Optimization and Ranking
The generative engine optimization study was limited to analyzing the top five positions in search results. The researchers measured how the optimizations impacted content in the different positions. Ultimately they found that optimizations had a greater impact on lower-ranking content’s ability to secure visibility in AI.
For example, Aggarwal et al found that the ‘Cite Sources’ method led to a significant increase in visibility for websites ranked fifth in SERP, while the visibility of the top-ranked website decreased, highlighting the differential impact of GEO based on SERP ranking
This raises a concern, getting back to how the authors minimize the role of SEO in AI visibility. If they are measuring visibility based on the top 5 results in the SERPS, these pages are very likely to have strong SEO as a foundation. If the AI is sourcing mainly from these positions, can one conclude that, contrary to the contention by the writers, SEO is playing a significant role in visibility?
Further, a just published study by Authoritas indicates that for commercial terms in Google SGE, “93.8% of the generative URLs do not match any URL from the first page of the organic search results.” The writers further state that “Google’s SGE often provides unique or additional content not found in the top organic results.”
This suggests that while SEO plays a role in visibility, AI systems like Google’s SGE are capable of sourcing content that might not traditionally rank high due to conventional SEO metrics. The authors may have overlooked the diverse ways in which AI systems prioritize the content they source.
Generative Engine Optimization Categories
The generative engine optimization study tagged queries according to 11 categories. These categories include:
- Debate
- History
- Science
- Business
- Health
- Statement
- Facts
- Law & Government
- People & Society
- Explanation
- Opinion
Methodology for Categorization
The authors used ChatGPT to create the query list and to categorize them. They state: ”To supplement diversity in query distribution, we prompt GPT-4 to generate queries ranging from various domains (eg: science, history) and based on query intent (eg: navigational, transactional) and based on difficulty and scope of generated response (eg: open-ended, fact-based.)”
The method raises concerns over the AI’s ability to categorize the terms correctly. It’s unclear whether the authors also used GPT to determine intent, difficulty, and scope, and if there was any human review of these classifications.
Clarity and Broadness of Categories
Concerning the specific categories, the ones the researchers used are quite broad. For example, the category “Business” is quite wide including B2B to B2C, services to products, and more. There are also questions about query intent, whether they were informational, commercial, navigational, or educational. Additional specificity would help others understand and test the study’s results in practical applications.
Missing Categories
Also, the generative engine optimization study is missing several categories relevant to website owners. Some categories omitted include:
- Entertainment and Media
- News
- Lifestyle and Wellness
- Education and Learning
- Travel and Geography
- Food and Cuisine
- Sports and Fitness
- Personal Finance and Investing
- Location-specific information
These missing categories raise the question: How applicable are the study’s findings to these areas?
Generalizability of Generative Engine Optimization
The generative engine optimization study relied on specific AI models and datasets. It used ChatGPT 3.5 Turbo to run its tests. As the authors state, “GPT-3.5 turbo was used for all experiments.” This raises questions about the applicability of the findings to other AI models and situations.
It’s possible, and very likely, that the algorithms used by ChatGPT, Claude, and Google SGE value different criteria for their responses. This suggests that their results may not be universally applicable.
Additionally, the study used a defined dataset in conducting its tests. As noted, “We evaluate all our methods on a subset of 200 samples of our test set.” The representativeness of this dataset in mirroring real-world user queries is important for determining the generalizability of their findings.
One of my concerns is that with Google being the biggest search engine and with Search Generative Experience available in test mode, it is curious that the researchers did not test their methods on it or Bing Chat. This raises even more questions about the generalizability of their conclusions.
User Engagement Metrics
This generative engine optimization study primarily focused on optimizing content for visibility in generative AI responses. This says nothing about user experience. it leaves unanswered several questions concerning user interactions with the responses. Are users engaging with the content by asking additional relevant queries about the topic?
Additionally, the study does not address user engagement metrics directly. Website owners typically want searchers to click on their search results. The study offers no insight into interactions with the AI responses.
The study did measure “Subjective Impression,” which included the relevance of the citation to query, the influence of citation on response, diversity, and uniqueness of information presented, the likelihood of follow-up by the user, and the amount of information presented in the answer. However, these measurements do not directly translate to user engagement metrics like clicks or additional queries.
Additional Strategies for Consideration
While the generative engine optimization study mostly focuses on strategies like quotes, authoritative writing, citations, and statistics, the are other strategies that should be explored for optimizing content visibility in AI. These include:
- Emotional Tones and Sentiment Levels. Investigate how different emotional tones (positive, negative, neutral) and varying sentiment levels in content influence AI visibility.
- Narrative Structures and Storytelling Techniques. Measuring how narrative structures and storytelling techniques affect visibility.
- Q&A Formatting. Can Q&A formatting improve content’s chances of being selected by AI for direct answers?
Multimedia Integration
if you’ve played around with Google’s Search Generative Experience, you’ve noticed that it builds responses using a variety of multimedia elements. The study does not address this.
In the screenshot below for “alpaca,” Google’s SGE provided a primary image and several images with links to their articles. Yes, I love alpacas!
The role of imagery, video, documents, and other media types in influencing AI visibility is not fully understood. This offers an important area for future study.
Long-Time Effectiveness of Generative Engine Optimization
The generative engine optimization study overlooks the likely changes in AI algorithms over time. This raises questions about the long-term viability of its findings. With refinement and evolution of the underlying algorithms, the strategies found successful at this moment in time may not retain effectiveness over time.
It’s imperative to understand the impact of the continual evolvement of algorithms in AI content optimization.
Rethinking the Name: Generative Engine Optimization (GEO)
The authors named the practice of optimizing content for AI Generative Engine Optimization, or just GEO. On a surface level, calling it “GEO” in an SEO context can be confusing as there already is a GEO SEO which is SEO focused on specific geographical areas.
Additionally, considering the rate of innovation, the term “generative engine” might also be too limited in scope and become outdated very quickly. Another term that might be more flexible is Artificial Intelligence Content Optimization or AICO.
The limitation of this term is its focus on “content.” There may be other aspects that need to be optimized for AI as well. For example, you may need to optimize the structure, design, functionality, and performance of your website to make it more compatible with AI platforms and devices.
Proposing AI Web Optimization (AIWO)
To be more wide-ranging, I suggest the term AI Web Optimization (AIWO), pronounced “Aye-Woh.”. This term would cover not only content but also other web elements and features that affect how AI interacts with your website.
This name would encompass:
- Content Structure and Quality
- Technical SEO for AI
- AI-friendly Data Formatting
- User Experience
- Ethical and Responsible Use of AI
- AI-Driven Analytics and Insights
- Accessibility
- AI-Powered Content Generation
- Mobile Optimization for AI
AIWO aims to provide a more holistic view of website optimization for AI.
Note 3/9/24: After some time and consideration, I’ve gravitated to the term Intelligent Search Optimization. Read more in my article, ” Will we still “Google It” in 2 years? Search in the AI Era.”
Final Thoughts
While the study “GEO: Generative Engine Optimization” provides a first step in empirically understanding optimizing for AI search visibility, it’s important to remember that this field is changing quickly. The findings from Aggarwal et al. provide a solid foundation to test in different contexts, but they are just the beginning.
To recap, for future research in AI visibility, I suggest:
- Understanding the Role of SEO
- Exploring Emotional Tones and Sentiment Levels
- Assessing Narrative Structures and Storytelling
- Q&A Formatting
- Multimedia Integration
- Long-Term Effectiveness Amid Algorithmic Changes
- Test Optimization Strategies Across a Variety of AI
- More Diverse Web Content Categories
- Broader Web Optimization for AI (AIWO)
If you like this, be sure to subscribe below!
Reference
Aggarwal, P., Murahari, V., Rajpurohit, T., Kalyan, A., Narasimhan, K. R., & Deshpande, A. (2023). GEO: Generative Engine Optimization. Retrieved from https://arxiv.org/pdf/2311.09735.pdf
