The benefits industry has reached peak AI noise. Every carrier, TPA, and technology platform has something with AI in the name or the pitch deck. Some of it is genuinely useful. Some of it is a workflow feature with a better marketing budget. And most brokers, when a vendor or colleague pitches an AI tool for their practice, do not yet have a clear way to evaluate it.
Eighty-five percent of brokers are now using or exploring AI tools, and the pace of adoption is accelerating.1 The push is coming from the top as well, with 87% of sales leaders saying their CEOs and boards are pressing for AI adoption.2 Adoption itself is a good thing. But adoption without evaluation is how practices end up with subscriptions nobody uses, implementations that stalled three months in, and a general sense that AI did not deliver on its promise when the real problem was never the technology.
What I have seen, watching AI implementations succeed and fail across our advisor network, is that the tools rarely fail because the technology does not work. They fail for much more predictable reasons, and the research backs that up. An MIT study found that about 95% of generative AI pilots at large companies were failing to produce measurable returns, largely because of how the tools were implemented rather than the technology itself.3 Most of those failures are avoidable.
THE PROBLEM GETS BOUGHT BEFORE IT GETS DEFINED
The most common failure mode in AI adoption is also the earliest one. A vendor demo is impressive. Leadership gets excited. A contract gets signed. And then, three to six months into implementation, someone finally asks the question that should have been asked first: what specific problem in our workflow are we actually solving?
AI tools adopted in response to enthusiasm rather than a defined need tend to have the same outcome. Low utilization, unclear ROI, and a lingering sense that the technology did not deliver, when in reality the technology was never given a real job to do.
Before committing to any AI tool for your practice, the more useful question goes beyond whether it works. Is the problem it solves worth solving in your specific workflow, at your current scale, right now? A tool that automates a process nobody finds painful adds a subscription fee without adding productivity.
THE DATA IS NEVER AS CLEAN AS THE DEMO
Every AI tool works better in a vendor demonstration than it does in a real deployment. That says less about vendors than about how demos work. The data is clean, the workflows are standardized, and the edge cases have been removed.
In practice, benefits data is messy. Census files have inconsistencies. Carrier requirements vary by market. Quote formats differ across carriers, creating real friction when you try to automate across them. The AI tool that produced beautiful outputs in the demo produces unreliable ones in production because the inputs are nothing like what the demo assumed.
THE TECHNOLOGY WORKS. NOBODY USES IT.
The second most common failure mode is not technical at all. The tool is implemented correctly. The outputs are accurate. And adoption across the team is 20% of what was projected, because nobody adequately prepared the people who were supposed to use it.
AI adoption fails on the human side far more often than the technology side. Sales or other stakeholders who were not involved in the buying process or design sessions. Producers who did not see the value and kept doing things the old way. Support staff who were not trained and defaulted to familiar processes. These are change management failures, not AI failures, but they get attributed to the technology every time.
Before any implementation, someone should be able to answer three questions. Who on the team is going to use this? Do they understand why it is better than the current process? And what does success look like for them in their day-to-day work, not for leadership or the vendor, but for the person actually doing the job? If those questions do not have clear answers, the implementation is not ready.
There is a personal side to adoption, too. The dopamine hit AI can deliver is real. Work that used to take hours gets done in minutes, and that near-immediate gratification can trick the brain into mistaking quick wins for genuine productivity. AI should further meaningful cognitive thought, not make us more dependent on the prompt-and-answer loop.
NOBODY ASKED WHO OWNS THE DATA
This is the failure mode that keeps me up at night, because it is the one that is hardest to undo.
Every AI tool in the benefits space runs on data. Client information, census files, quote data, enrollment records. The security and ownership of that data carries real compliance implications, and most practices do not dig into those implications until something goes wrong.
Fiduciaries and advisors evaluating AI-driven tools should understand how the AI models are built, what data they use, how results are validated, and whether cybersecurity and privacy controls are adequate.4 A tool that processes sensitive client data without clear contractual protections around ownership, model training, and breach notification is a liability risk regardless of how well it performs.
The specific questions worth raising before any contract is signed: Who owns the data once it enters the platform? Is it used to train or improve the model? What happens to it if the relationship ends? Is the vendor compliant with applicable state privacy laws? These are due diligence, not hostile questions.
Regulators are paying attention, and the common thread across their rules is human oversight. Illinois now requires employers to give notice when AI is used in certain employment decisions and bars uses that result in discrimination.5 Colorado shows how fast the ground can move. Its original AI Act would have required impact assessments and risk management programs, but in 2026 the state replaced it with a narrower, disclosure-based law that takes effect January 1, 2027, joining a patchwork of state and local rules that includes New York City’s requirements for automated hiring tools.6 At the federal level, a 2025 executive order pushes toward a single national framework and directs challenges to state AI laws, but it does not erase those laws, and existing antidiscrimination laws still apply.7 While the rules keep shifting, a qualified human reviewing any AI-supported decision remains the most durable protection.
BOTTOM LINE
AI will not simplify your practice by accident. It pays off when adoption decisions rest on evidence rather than enthusiasm. That kind of disciplined thinking draws on operational judgment more than technical skill, and that judgment is where a durable advantage gets built.
CRC Benefits is investing in AI partnerships and platform solutions, and these are the same questions we ask before bringing any of them to advisors: whether it solves a real workflow problem, how it handles real-world data, how it protects that data, and whether people will actually use it. That way, the technology you bring to clients has already been tested against the places these tools tend to break.
Reach out to the CRC Benefits team to see what that looks like in practice.
CONTRIBUTOR
- Katie Crist is Vice President of Product Development for CRC Benefits, where she leads the product and technology strategy behind the company’s advisor support. She focuses on putting AI and new technology to work in ways that hold up in real workflows, with strong governance and the people using the tools always in view.
END NOTES
- iSolved. “Emerging Trends for Brokers in 2026.” January 2026. Via PLANADVISER: https://www.planadviser.com/brokers-increase-ai-use-as-compliance-demands-grow/
- Gartner. “Top 3 Strategic Priorities for Sales Operations Leaders.” https://www.gartner.com/en/sales/insights/top-3-strategic-priorities-for-sales-operations-leaders/unlocked
- Forbes. “MIT Finds 95% of GenAI Pilots Fail Because Companies Avoid Friction.” Aug. 26, 2025. https://www.forbes.com/sites/jasonsnyder/2025/08/26/mit-finds-95-of-genai-pilots-fail-because-companies-avoid-friction/
- Benefits Law Advisor. “Harnessing AI Under ERISA: A Compliance and Oversight Guide.” July 2025. https://www.benefitslawadvisor.com/2025/07/articles/ai/harnessing-ai-under-erisa-a-compliance-and-oversight-guide-for-retirement-and-health-plan-fiduciaries/
- Seyfarth Shaw. “Artificial Intelligence Legal Roundup: Colorado Postpones Implementation of AI Law as California Finalizes New Employment Discrimination Regulations and Illinois Disclosure Law Set to Take Effect.” 2025. https://www.seyfarth.com/news-insights/artificial-intelligence-legal-roundup-colorado-postpones-implementation-of-ai-law-as-california-finalizes-new-employment-discrimination-regulations-and-illinois-disclosure-law-set-to-take-effect.html
- Troutman Privacy. “Colorado Legislature Passes Bill to Repeal and Replace Colorado AI Act.” May 2026. https://www.troutmanprivacy.com/2026/05/colorado-legislature-passes-bill-to-repeal-and-replace-colorado-ai-act/
- Fisher Phillips. “Trump Signs AI Executive Order Targeting State Laws.” December 2025. https://www.fisherphillips.com/en/news-insights/trump-signs-ai-executive-order-targeting-state-laws.html