ClaudeBuilding the best agentic analytics harness: Powered by Claude, built with Claude Code
CHAPTERS
How Claude changed Omni’s engineering velocity
Chris Merrick (Omni CTO) explains how adopting Claude—especially Claude Code with Opus—materially increased engineering throughput. He highlights that even as CTO he can still contribute meaningful code, and shows how the team’s commit velocity ramped after the tooling “clicked.”
Operational culture: “Ship it,” transparency, and public demos
Omni’s velocity is reinforced by rituals and transparency, including weekly all-hands demos recorded and shared publicly. This practice helps customers and the community see how Omni builds and iterates in real time.
What Omni built: AI analytics that translates questions into executable queries
Chris outlines Omni’s core AI analytics workflow: users ask natural-language questions, Claude translates them into semantic queries, and Omni executes them via a semantic layer against the data warehouse. The key challenge is embedding business-specific meaning so answers match company definitions.
Why the semantic layer matters: curation, context localization, permissions, and feedback
The semantic layer is presented as the essential bridge between messy real-world warehouses and reliable analytics. It curates relevant datasets, localizes context next to definitions, enforces permissions, and supports a continuous improvement loop based on usage.
Meet Blobby: the agent analyst and its real user experience
Chris introduces Omni’s AI agent, Blobby, and shows the end-to-end experience: interpreting user intent, mapping terms to the semantic model, selecting filters, generating/executing queries, and presenting visualizations plus narrative summaries.
Blobby v1 lessons: adding AI-specific metadata (context, samples, values)
Early Blobby handled single questions with limited guidance, so Omni expanded metadata to steer the model. They added LLM-targeted context, sample queries for grounding, and representative field values to improve filter selection and terminology mapping.
Making it agentic: building an internal agent harness with tasks and error recovery
Omni then wrapped Blobby with an agentic loop—introducing tasks and iterative execution. A major quality jump came from teaching Blobby to recover from errors and investing in descriptive error messages that guide retries and fixes.
Scaling capability: Haiku → Sonnet, higher token usage, and customer adoption
As conversations became more complex, Omni moved from Haiku to Sonnet to support deeper agentic interactions. Token usage rose by design, and customers reported major time savings and previously infeasible analyses completed in minutes.
“Blobotomies”: tracing failures and fixing architecture with a consolidated brain
Pressure from the CEO to make the system more predictable led Omni to invest in tracing and observability. Trace analysis revealed architectural issues—especially a split between an outer planner and a query sub-agent—so they consolidated capabilities to eliminate ‘split-brain’ behavior.
Unlocking expressive SQL: dusting off a parser and shifting from JSON queries to SQL
Omni noticed Claude could sometimes write better SQL than Blobby’s structured query interface allowed. They revived an older SQL parsing engine and changed the interface so Blobby could output SQL directly (within constraints), enabling more powerful one-shot queries (often using CTEs).
Today’s system: outer checkpointing loop + expanding toolset + evals for predictability
Chris describes the current architecture: an outer loop for checkpointing and recovery, and an inner loop with a growing toolkit (dashboards, visualizations, validation, semantic modeling). Evals are central—especially for trace observability—and Omni is extending eval capabilities to customers.
Building better harnesses by being power users of Claude Code
Omni’s team learned harness design by using Claude Code themselves, improving empathy for user workflows and understanding what “good” feels like. They draw parallels between navigating a codebase and exploring a semantic model, borrowing proven interaction patterns.
Live demo: auto-generating an engineering activity dashboard and refining via UI
Chris demonstrates Blobby creating a dashboard from a prompt, including planning and topic selection. He then shows how users can open results in a workbook to inspect filters, adjust repositories, and troubleshoot issues—illustrating the AI-to-UI refinement workflow.
Wrap-up: Omni powered by Claude and optimized for the Claude model family
Chris closes by reiterating that Omni’s agentic analytics harness is designed around Claude’s strengths. He notes strong customers, a high-performing team in San Francisco, and invites attendees to connect (and grab a Blobby sticker).
