by Amy Hebdon for Search Engine Land
Have you tried to use generative AI in your digital marketing but found that it behaves more like “spicy autocomplete” than actual intelligence?
Try using disconfirming questions. Here’s how.
The power of disconfirming questions
Disconfirming questions challenge what you already believe or think is true.
Instead of asking questions that agree with your ideas, disconfirming questions encourage you to question and explore different possibilities.
Contrast this with confirming questions, which seek validation and reinforce existing perspectives.
Notice the difference between these two questions and the answers they’ll elicit:
- Confirming: Do you agree this new feature will greatly enhance the user experience?
- Disconfirming: What potential drawbacks or challenges might arise from implementing this new feature, and how might they impact the user experience?
How you ask a question has a big impact on the response.
In a Wharton study, participants were asked to play the role of a salesperson selling a used iPod. They were told the iPod had crashed twice in the past, wiping out all the music stored in it.
The researchers wanted to see what types of questions would lead the sellers to disclose the problem. Here are the questions from the experiment:
When asked, “What can you tell me about the iPod?” fewer than one in 10 sellers disclosed the problem. More than 10x as many sellers disclosed the crash when asked, “What problems does it have?”
I borrowed this experiment to see how ChatGPT would change its responses based on the questions I gave. Here’s how it went:
Prompt A: “I would like to buy this used iPod. What can you tell me about it?“
ChatGPT response: “…iPod models vary in terms of features, storage capacity, and generation. Please provide me with the necessary details or any identifying information about the iPod so that I can assist you better.”
After confirming that it’s a language model and informing me it would need more specific details, the response is a vague overview of the differences between iPod models.
Compare that answer to the one it provides for a disconfirming question:
Prompt C: “I would like to buy this used ipod. What problems does it have?“
ChatGPT response: “…I don’t have access to specific information about the individual iPod you’re referring to. However, when buying a used iPod, it’s important to consider potential issues that could arise. Here are some common problems that could affect a used iPod:
| Potential Issues | Description |
| Battery Life | The battery may not hold a charge as well as it did when the iPod was new, resulting in shorter battery life and more frequent charging. |
| Physical Condition | Used iPods may have scratches, dents, or other signs of wear and tear. It’s important to assess the physical condition of the device through pictures or inspection. |
| Screen Issues | The iPod’s screen may have dead pixels, discoloration, or backlight problems. Check for any visible screen defects or irregularities. |
| Functionality | Verify that all buttons, ports, and features of the iPod are working correctly. Test the headphone jack, volume buttons, and other essential functions. |
| Software and Compatibility | Ensure the iPod is compatible with the software and operating system versions you plan to use. Older iPod models may not support the latest updates or apps. |
| Storage Capacity | Confirm the exact storage capacity of the iPod and ensure it matches your requirements. Verify that the advertised storage size is accurate. |
To minimize the risk, consider purchasing from reputable sellers with good ratings or warranties, if available. Additionally, ask the seller about any known issues or defects the iPod may have before making a final decision.”
The second prompt didn’t provide any extra information, but the response gave me an action plan to ensure I get a working iPod.
Much like the sellers in the original Wharton study, ChatGPT seeks to answer the specific questions we ask.
This means we can influence the quality of responses we get based on the prompts we provide.
Asking disconfirming questions to generative AI can result in more transparent and useful responses.
Get the daily newsletter search marketers rely on.SUBSCRIBE
How disconfirming questions can benefit your digital marketing strategy
While disconfirming questions are useful in most professions, they’re especially valuable in digital marketing, where the volume of data and lurking variables can easily lead to the wrong conclusions.
Using disconfirming questions in your marketing strategy can help you:
- Avoid confirmation bias by actively seeking evidence that contradicts your assumptions or initial interpretations.
- Uncover hidden patterns by digging deeper, exploring outliers, and considering alternative perspectives.
- Test assumptions by adopting a more rigorous approach to data interpretation.
All this drives more accurate data and conclusions, leading to better results.
Building a practice of asking high-quality disconfirming questions also contributes to professional growth, including:
- Critical thinking skills: Develop a more balanced and objective perspective as you seek ideas and viewpoints that contradict your current understanding.
- Breakthrough innovation: Challenge conventional wisdom and question the status quo to find innovative insights that would have been overlooked.
- Strengthened problem-solving abilities: Build a more holistic and complete analysis of a situation to identify new angles and potential solutions.
When paired with generative AI, the benefits of disconfirming questions are even more powerful.
That’s because you can use generative AI as a brainstorming partner, giving you guidance and answers to otherwise rhetorical questions. AI can help you:
- Identify diverse perspectives: Generative AI can provide fresh and diverse perspectives you may not have considered.
- Overcome mental blocks: Get help with new ideas, alternative viewpoints, and unexpected connections.
- Collaborative iteration: Use AI iteratively, ask follow-up questions and challenge responses, and bring your own judgment and expertise to the conversation for a productive and interactive dialogue.
Let’s look at specific digital marketing areas where you can combine disconfirming questions and generative AI for powerful results.
How to uncover untapped marketing opportunities with disconfirming questions
To make the most of your prompts and questions, it’s important to start by clearly understanding the problem. When problems are well-defined, they become easier to solve.
So, before creating your prompts and questions, take the time to grasp the problem and its details fully. This will help you develop effective prompts and questions that lead to meaningful solutions.
Landing page disconfirming questions
Your PPC ads’ success depends on a high-converting landing page, but landing page development is often a different discipline than digital marketing.
If you work on landing pages, these questions can help your process.
If you aren’t directly responsible for landing pages or don’t feel comfortable weighing in on edits, you can provide high-quality feedback as a value-add.
Here’s how to use generative AI to identify improvements to increase conversion rates.
First, paste your landing page content into the prompt box, or use a plugin to connect to the page directly. Then ask disconfirming questions, such as these sample prompts:
Prompt: “What is missing from the landing page that might keep a customer from buying?”
ChatGPT can provide feedback on specific elements (such as product descriptions or CTA buttons), usability and navigation, value propositions, social proof and trust signals, and persuasive content.
Prompt: “What elements or features of my landing page could be confusing or misleading to visitors, and how can I clarify them?”
ChatGPT can analyze your page’s clarity of messaging, message hierarchy, navigation, consistency and trustworthiness, and the potential of information overload and provide recommendations for improvement.
Audience disconfirming questions
Many prebuilt prompts for audience insights address strictly demographic data, such as asking about the age range of your target market.
What if instead you ask disconfirming questions like:
Prompt: “What are some potential biases or assumptions that I may have about my audience, and how can I challenge them?”
ChatGPT can help identify potential behavior assumptions about your audience, such as their decision-making process, cultural or linguistic biases, psychographic assumptions (like motivations and needs), and demographic assumptions.
Prompt: “What are some reasons why my audience might not be interested in my product or service, despite its benefits and features?”
ChatGPT can help you consider factors like misaligned value propositions, brand awareness (or lack thereof), the effectiveness of targeting, competitive landscape, communication gaps, and price and affordability.
You can also use disconfirming questions to identify new target audiences (“Could there be a different demographic group that would benefit from our product/service that we haven’t considered?”), or to evaluate specific audience segments (“How might our marketing messages be perceived differently by diverse segments of our audience?”).
Client disconfirming questions
Managing digital marketing and managing client expectations are two different skill sets. Fortunately, ChatGPT can help with both.
Prompt: “What are some potential objections or concerns my client may have about my marketing strategies, and how can I address them in a clear and effective manner?”
ChatGPT can help anticipate and address issues your clients may have with strategy, messaging, measurement, reporting, positioning, brand identity, or budgets.
Prompt: “Can you think of any reasons why my client may not be satisfied with my current communication approach, and how can I improve it?”
No one’s a perfect communicator. ChatGPT can reveal issues with clarity, frequency, personalization, channel selection, feedback methods and use of visuals to make your client communication more effective.
Driving innovation in ChatGPT with disconfirming questions
The best way to use generative AI is through collaboration.
Combine your own knowledge, creativity and experience with the power of disconfirming questions to drive high-value discussions, get better results and grow professionally.
