Transforming Dashboard Intelligence with Agentic AI
Due to NDA, some content and specifics have been redacted.
Scroll to read the case study
The problem
Our client’s internal employee survey found an interesting fact: users were spending real time just finding the right answer to their question, and that alone was hurting their overall efficiency. Claims analysts and claims managers were going back and forth between the database and existing reports, which led to inconsistent numbers and even more time spent chasing an answer; not to mention multiple copies of the same report, each built to serve a different person’s version of the question.
Executives are forced to make critical, time-sensitive business decisions based on outdated or surface-level data. When existing dashboards lack specific answers, they face a lose-lose choice: wait days or weeks for data engineering or BI teams to build a custom report, or spend precious executive hours manually hacking together fragmented data in Excel.
Analysts spend 80% of their time hunting and stitching data across dozens of disparate systems (and verifying business logic with domain experts) rather than actually analyzing it. When an anomaly occurs, finding the true root cause requires tedious, manual query-building across unlinked databases, under immense pressure to report back to executives.
Reframing the goal
The first instinct was to fix the dashboard with better filters, a cleaner view, maybe a saved report for the region in question. But the real gap wasn’t visibility, it was translation: someone making a high-stakes decision doesn’t need another chart, they need someone to read the chart for them, in the few minutes before they have to act on it. That’s not a dashboard problem. It’s a reasoning problem which meant the fix couldn’t be a better view of the data. It had to be something that could look at the data, form a judgment, and say so in plain language.
Why agentic AI
Context: the agent can handle the context of a conversation on a much larger scale.
Domain-specific knowledge: we can ground it in the knowledge and context of the domain insurance, in this case specific to the client and their environment.
User-persona-specific guardrails: guardrails tailored to generate specific output based on the input are crucial, and agentic AI handles this far better than a single model.
Data handling: federated multi-source orchestration across 100+ live datasets, without building new ETL pipelines.
Adaptability: dynamic query execution via a DB Code Interpreter, to compute net-new logic on the fly.
Problem solving: multi-step reasoning that explains why something happened and recommends a next step — root cause analysis, not just a number.
Building the agent
My skills in a nutshell:
The 3 pillars
Primarily designed for claims analysts to find specific data points and the reasons behind any changes in them.
Designed for claims managers who need a level up from an analyst’s view like a dashboard, but as a proactive conversation instead of static visuals, with on-the-go dynamic visuals to make sense of related data points.
Built for executives who need the headline summary of the data points, and what teams are doing in response.
Important guardrails
Agent orchestration
Agent workflow
Two layers make up the workflow: how the agent carries a conversation with the user, and what happens underneath each turn to actually produce an answer.
Validation was done in iterative loops. The process: problem identification, justification of a model, a user story for the agent specific to each persona, and continuous evaluation applying guardrails and controls to reduce hallucinations. I used the HHH (Honest, Helpful & Harmless) evaluation method to verify model performance.
Designing the experience
The user selects their own path to get to their findings. Below is the snapshot of Root Cause Analysis flow.
Journey map: Nancy, VP of Claims Strategy
Impact
Learnings
Building the agent taught me as much about restraint as it did about AI. The guardrails capping what the agent could claim, requiring it to show its work, keeping a human in the loop for anything consequential took longer to get right than the orchestration itself, and they mattered more. It’s easy to make an agent sound confident; it’s harder to make it trustworthy enough that someone would act on its word in a high-stakes moment. That’s the real design problem in agentic AI: not the interface, but the judgment behind it.
