What Smart Leaders Are Asking Before Investing in AI (Version A — 7 Diagnostic Questions)
What smart leaders are asking before investing in AI (Version A — 7 Diagnostic Questions): the key questions that help separate high-impact AI initiatives from expensive experiments.
ARTIFICIAL INTELLIGENCE
Video Guru
6/27/20268 min read
The board meeting ends. Someone says, "We need an AI strategy." Everyone nods. Nobody asks what that actually means. Six months and seven figures later, the company owns an expensive chatbot twelve people use and a machine learning model that predicts last quarter's results with impressive accuracy.
Leaders jump to solutions before diagnosing the problem. They write checks before answering the questions that determine whether those checks buy anything more than consultant comfort.
What follows is a diagnostic framework — seven essential AI strategy questions for CEOs — that I've developed through years of advisory work. These are practical gates that separate productive investment from expensive theater. Each maps to one dimension of the S•I•C•T framework — Structure, Information, Cohesion, and Transformation — the lens I use to assess organizational readiness before any technology decision.
Use this as a self-assessment. The gaps you discover aren't failures — they're the map to where your investment should actually go first.
Question 1: What Business Problem Are We Actually Solving? (Structure)
This sounds obvious. It isn't.
Most AI initiatives start with the technology, not the problem. "We want to use AI for customer service" isn't a problem statement — it's a solution looking for a home. The right framing: "Our team spends 40% of their time on repetitive inquiries, and resolution times are costing us renewals."
The diagnostic test: Write your AI initiative on a whiteboard. Erase the word "AI" and every mention of technology. What's left? If you can't articulate a specific business outcome in plain language, you don't have a use case — you have a preference.
Structure, the organizational architecture dimension of S•I•C•T, demands you know what piece of your operation you're improving before selecting the tool. If your team can't state the specific metric that will move — revenue per employee, error rate, cycle time — you're not ready to invest. You're ready to investigate.
Question 2: What Data Exists, and Can We Trust It? (Information)
AI systems learn from data. Yet I consistently encounter organizations that want to build predictive models on datasets that are incomplete, inconsistently labeled, or unreliable. The excitement about capabilities eclipses the reality of requirements.
Your AI strategy is only as good as your data infrastructure. If your customer data lives in seventeen systems, if your financial records require manual reconciliation, if nobody can agree on how key metrics are calculated — these aren't minor inconveniences. They're structural blockers that will absorb your AI budget before a single model gets trained.
The Information dimension of S•I•C•T is about signal versus noise. Most organizations assume their data problem is volume. Usually, it's quality. Before you invest, audit your data assets honestly. The gaps here will tell you where your first dollars should actually go — and it might be data engineering, not machine learning.
Question 3: Who Owns This Decision — and Who Owns the Outcome? (Structure)
AI initiatives fail at the intersection of authority and accountability. I've seen IT departments select tools that operations never asked for. I've seen impressive pilots die in production because nobody was accountable for scaling them.
Before you invest, be crystal clear: Who has decision rights? Who will be measured on outcomes? Who controls the budget, defines success criteria, and owns change management? If your answer is "a committee," you have a governance problem. If it's "the CTO" but outcomes affect sales and operations, you have an alignment problem.
The Structure dimension of S•I•C•T focuses on this precisely — roles, decision rights, and organizational architecture. Smart organizational design isn't about approval layers. It's about ensuring decision-making authority sits with people who understand both the technology's capabilities and the business's constraints. One clear owner beats a consensus committee every time.
Question 4: Are Our Teams Ready to Work With AI? (Cohesion)
Technology is easy. People are hard.
The Cohesion dimension of S•I•C•T measures something most technology assessments ignore: whether your culture can absorb and leverage new capabilities. I've seen technically perfect AI implementations fail because managers undermined them, teams hoarded data, or informal power structures treated AI as a threat.
This isn't about training — it's about readiness. Do your people understand what AI can and cannot do? Are managers prepared to lead teams where human judgment and machine prediction work together? If your organization punishes failed experiments, no AI strategy will overcome that cultural antibody.
The honest assessment: Would your best people embrace AI tools that make their jobs different, or would they preserve the status quo? Their answer determines whether your investment multiplies or evaporates.
Question 5: What's the Minimum Viable Proof Point? (Transformation)
CEOs love transformative visions. But transformation without validation is expensive hope.
The Transformation dimension of S•I•C•T isn't about speed — it's about learning velocity. Can your organization test hypotheses quickly, learn from evidence, and adapt? Before committing to enterprise-scale deployments, you need a proof-of-concept strategy that validates core assumptions at minimum cost.
Define the smallest version of your initiative that produces meaningful evidence. Not a pilot — pilots become zombie projects that never die but never scale. A proof point with clear success criteria, a 60-90 day timeline, and an explicit decision gate: scale, pivot, or kill.
The leaders who get ROI from AI aren't the ones with the biggest budgets — they're the ones with the fastest learning cycles. The quality of your decision-making matters more than the quantity of your AI spending.
Question 6: How Will We Measure Whether This Worked? (Information)
"We'll know it when we see it" is not a measurement strategy.
Before you spend a dollar, define success in specific, measurable terms. Revenue impact? Cost reduction? Error rate improvement? Pick your metrics. Establish your baseline. Set your targets.
The Information dimension of S•I•C•T emphasizes that good decisions require good feedback loops. Most AI initiatives track activity ("we deployed the model") rather than outcomes ("we reduced processing time by 34%"). Without clear metrics, you can't optimize. You can't even evaluate whether the investment was worth it.
If you can't define success precisely enough that an objective third party could verify it, you're not ready to invest. You're ready to think harder.
Question 7: What Are We Willing to Stop Doing? (Structure)
Resources are finite. Attention is scarcer. Every AI initiative competes for both.
The most honest question is also the least asked: What existing project or priority gets reduced or delayed to make room for AI? If the answer is "nothing — we'll just add this on top," you're planning to fail.
The Structure dimension of S•I•C•T includes resource allocation as core governance. Effective AI strategy requires portfolio thinking. Which initiatives matter most? Which have the best risk-adjusted returns? The discipline to say no — to deprioritize the interesting for the important — separates organizations that succeed with AI from those that accumulate disappointments.
This is where strategic clarity becomes operational reality. Your AI priorities should emerge from a clear-eyed assessment of where you are and what you can execute well — not from competitive anxiety or boardroom pressure.
The Self-Assessment Scorecard
Rate your organization on each question from 1 to 5:
Score
Meaning
1
No clear answer; significant gap
2
Partial answer; known issues
3
Reasonable clarity; some gaps
4
Clear answer; minor issues
5
Strong clarity; execution-ready
Total score interpretation:
28-35: Strong position. Proceed with confidence, but validate with a focused proof-of-concept.
20-27: Foundation exists, but gaps need attention before major investment.
14-19: Significant readiness gaps. Invest in preparation — data, governance, change management — first.
7-13: Fundamental prerequisites missing. Focus on organizational and data readiness before AI-specific spending.
The score isn't a judgment. It's a navigation tool that tells you where to invest first so your AI investments produce real returns.
The Bottom Line on What to Ask Before AI Investment
I've watched companies spend millions on AI initiatives that produced nothing because they skipped the diagnostic work. The problem wasn't the technology — it was organizational: unclear problems, unreliable data, ambiguous ownership, unprepared teams, undefined metrics, and no resource discipline.
The companies that succeed do the hard thinking first. They treat AI as a business capability, not a technology purchase. They invest in readiness before solutions.
If you're a CEO considering AI investment, these seven questions are your CEO AI checklist. Answer them honestly. Address the gaps you find. Then invest with confidence that you're solving real problems with real capabilities, not performing innovation theater.
The leaders who get this right aren't the ones with the biggest budgets. They're the ones who asked the hardest questions first.
Frequently Asked Questions
1. How long should a proper AI readiness assessment take?
A thorough diagnostic takes 3 to 6 weeks, depending on organizational complexity. This isn't about creating a lengthy report — it's about having the right conversations and reviewing your data infrastructure honestly. The assessment should produce a clear prioritization of gaps and a sequenced plan — not a binder that sits on a shelf.
2. Should we hire AI specialists before conducting this assessment?
Generally, no. The AI readiness assessment is about understanding your organization's readiness and identifying the right problems to solve. Bringing in technical specialists too early means they'll build solutions before you've defined the right problems. Start with strategic clarity. If you need external help, look for advisors who understand both business and technology dimensions — people who can ask hard questions about your operations, not just recommend tools. I conduct these assessments through Roth AI Consulting because the diagnostic phase requires business judgment, not just technical expertise.
3. Which of the seven questions is most commonly neglected?
Question 7 — what are you willing to stop doing — gets skipped most often. Leaders hate making explicit trade-offs. The question about data quality (Question 2) is also frequently underestimated. Most companies overestimate their data readiness because they haven't actually tried to use that data for automated decision-making.
4. How do we handle board pressure to "do something with AI"?
Reframe: "We're taking a disciplined approach to ensure our AI investments produce real returns." Use this seven-question framework as your methodology. Boards love rigor — what they hate is writing checks and watching nothing happen. Show them a structured assessment with clear gates and decision points. Rushing into visible but poorly planned initiatives creates bigger credibility problems when they fail.
5. Can smaller organizations use this same framework?
Absolutely. Smaller organizations often have an advantage — fewer silos, simpler data environments, faster decision cycles. The framework scales down naturally; you just adjust the depth of analysis. A 200-person company might complete this in two weeks. The fundamental questions don't change because organizational physics don't change.
6. What if our assessment reveals we're not ready for AI at all?
That's valuable information, not a failure. Many organizations discover that near-term investments should go toward data infrastructure, process documentation, or cultural readiness — not AI directly. Use the assessment to build a readiness roadmap. Address foundational gaps first, then revisit AI initiatives when scores improve. This is how smart leaders allocate capital — toward preparation that enables future returns.
7. How do we prevent AI pilots from becoming zombie projects?
Every pilot needs a charter with three elements: clear success criteria, a hard timeline, and a decision gate. At the end, the decision is binary: scale, pivot, or kill. No "extend and see." The biggest reason pilots become zombies is that nobody wants to admit something didn't work. Build the kill option into the charter from day one, and assign an executive sponsor with authority to make the hard call.
8. How does AI strategy connect to broader digital transformation?
AI strategy should be a component of overall transformation, not a separate initiative. If you have a digital transformation program, the AI assessment should integrate into it — same problems, same governance, same priorities. If you don't have a broader program, that's a warning sign. AI doesn't work well in isolation. It requires the same foundations: clean data, modern infrastructure, process clarity, cultural readiness. Organizations that succeed think holistically, not in technology silos.
9. Should we build AI in-house or work with vendors?
For most organizations, the right approach is hybrid: buy foundational capabilities (cloud infrastructure, pre-trained models) and build what's specific to your business — your data pipelines, integration logic, domain applications. Building everything from scratch is wasteful. Buying everything off-the-shelf forces your business to fit someone else's assumptions. The assessment clarifies where your distinctive capabilities are — that's what you build. Everything else, you buy or partner for.
10. How often should we revisit this assessment?
I recommend a full reassessment annually, with a lighter quarterly check-in on your three lowest-scoring dimensions. The leaders who treat AI readiness as a dynamic capability rather than a one-time checkbox maintain advantage as the competitive environment evolves. The framework stays constant; your scores should move as you invest in the right foundations.