How I AIClaude Code + 15 repos: how a non-engineer answers every customer question | Al Chen
CHAPTERS
- 0:00 – 4:21
Why docs and generic AI answers failed for real customer questions
Al explains the moment he realized public documentation—and even AI summarizing those docs—couldn’t answer the nuanced, systems-level questions customers were asking. The gap wasn’t “what does the feature do,” but how multiple services interact in a specific deployment reality.
- •Enterprise customers ask deep, implementation-level questions
- •Public docs don’t capture cascading service interactions
- •LLM answers based only on docs still miss customer needs
- •Al’s background: not an engineer, but needs technical depth for the role
- 4:21 – 4:51
Mapping the platform: many services, many repos, one customer problem
They walk through Galileo’s architecture: multiple Kubernetes-deployed services, each in its own repository. That multi-service, multi-repo structure is exactly why customers’ questions span components and why single-doc answers fall short.
- •Galileo is an observability platform for AI apps
- •Architecture consists of many deployable backend services
- •Each service corresponds to a separate repo (not a monorepo)
- •Customer questions often require cross-service reasoning
- 4:51 – 6:46
The key unlock: pull ~15 repos into one VS Code workspace
Al describes consolidating all repositories into a single parent directory in VS Code so Claude Code can traverse everything. This reduces dependency on engineers for answers and lets him learn the codebase as he supports customers.
- •All repos cloned under one root folder for unified search
- •Claude Code can answer questions across repos
- •Prompts can explicitly direct Claude to check multiple services
- •Result: fewer Slack pings to engineering and faster customer responses
- 6:46 – 8:32
Why “current code” beats documentation as source of truth
Claire highlights that customers want step-by-step, present-tense explanations—not static docs. Because code changes frequently, the main branch becomes the most reliable reference for how the product works today.
- •Customers want detailed mechanics, not high-level docs
- •Docs and tribal knowledge drift as code evolves
- •Main branch is the best available source of truth
- •Querying code enables accurate, up-to-date explanations
- 8:32 – 9:54
Keeping everything updated: the 16-line ‘pull all repos’ script
Al solves the practical pain of updating many repos by asking Claude Code to write a small script that pulls the latest main branch everywhere. This makes “code as truth” viable day-to-day without manual git busywork.
- •Manually running git pull in 15 repos doesn’t scale
- •Claude Code generates a short script to update all repos
- •Running it daily keeps local context fresh
- •Enables reliable answers aligned with the latest releases
- 9:54 – 11:47
Opening at the right level: multi-repo context in the IDE
They emphasize a simple but underused tactic: open the workspace at the parent directory level so the assistant can traverse repos. The right scope (narrow vs broad) depends on the question you’re trying to answer.
- •Open the project at the directory level that matches the problem scope
- •Multi-repo workspaces reduce back-and-forth querying per repo
- •Tradeoff: broader scope can introduce context bloat
- •Technique applies to VS Code, Cursor, and similar IDEs
- 11:47 – 13:15
Live workflow: a deployment command that consults Confluence and code
Al demonstrates a custom Claude Code command for deployments that first searches Confluence (via Atlassian MCP) and then falls back to code when needed. The output is a step-by-step plan tailored to the customer’s constraints.
- •Custom command for frequent tasks (e.g., deployments)
- •Uses Confluence as first-stop guidance via MCP
- •Traverses repos for details not present in docs
- •Designed to produce practical, step-by-step deployment instructions
- 13:15 – 15:02
‘Customer quirks’ knowledge base: tailoring answers to each enterprise environment
Al maintains an evolving Confluence page capturing each customer’s unique security and infrastructure constraints. Claude uses this to generate highly customized responses that build trust and reduce generic, unhelpful guidance.
- •Enterprise deployments include air-gapped and security-heavy constraints
- •A single ‘customer quirks’ page stores per-customer requirements
- •Claude incorporates quirks to tailor deployment instructions
- •Customized answers outperform generic Kubernetes/infra guidance
- 15:02 – 17:04
Living with more chaos: AI as the cross-system navigator
Claire reframes “source of truth” debates: with AI able to traverse Confluence, Slack, Notion, and code, teams can be less precious about perfect organization. Al reinforces: capture useful context anywhere, then retrieve it via MCPs.
- •AI reduces the need for perfectly curated documentation systems
- •Useful context can live in Slack threads, Confluence, Notion, etc.
- •Slack MCP can summarize and extract knowledge from conversations
- •The goal shifts from organizing to making context retrievable
- 17:04 – 18:22
Competing on customer experience (not just shipping velocity)
Claire argues AI isn’t only a product-velocity tool; it’s a customer experience advantage. When responses are specific, accurate, and pre-tailored, customers perceive a stronger partnership beyond the product itself.
- •AI can differentiate companies through better customer interactions
- •Tailored deployment guidance increases trust and satisfaction
- •Customer-facing excellence becomes a competitive moat
- •This approach reduces time-to-value for enterprise deployments
- 18:22 – 20:31
Should customers query proprietary code directly? The ‘Al bot’ thought experiment
Al explores the logical next step: if he can answer by querying code, why can’t customers do it themselves? They discuss constraints like proprietary code, and contrast with open-source ecosystems where self-serve code querying is feasible.
- •Al acts as a bottleneck because he has code access
- •Idea: provide a sanitized/restricted code view for customers
- •Open-source examples (e.g., LangChain) enable direct repo querying
- •Tension between transparency, IP protection, and support scalability
- 20:31 – 26:21
Where humans still add value: editing, judgment, and validating with engineers
Al explains he doesn’t paste AI outputs verbatim; he edits for brevity, relevance, and tone. He also sanity-checks uncertain answers with engineers, noting some truth lives in plans, meetings, and hallway conversations—not in code.
- •Human role: proofread, simplify, and tailor to what the customer actually needs
- •AI outputs can be too verbose or ‘bot-sounding’
- •Verification: confirm tricky details with engineering to avoid hallucinations
- •Some critical context isn’t captured in repos (future refactors, internal discussions)
- 26:21 – 29:18
Reactive Slack support at scale: turning threads into knowledge base articles (Pylon)
Al shows how Galileo supports customers in Slack and uses Pylon to draft help articles from long threads. This converts one-off support into reusable, public knowledge that’s often more current than polished docs.
- •Enterprise support happens in shared Slack channels
- •Pylon monitors channels and helps convert threads into articles
- •Articles are abstracted to remove customer-specific details
- •Knowledge base becomes a fast, living layer alongside official docs
- 29:18 – 32:09
The ‘and then’ virtuous cycle: from one question to enablement, SEO, and roadmap insights
Claire describes “and then” workflow discovery: once an answer exists, AI makes it cheap to transform it into documentation, training, and growth assets. Al adds that clustering customer questions can inform product roadmaps with far less manual work.
- •AI lowers the cost of chaining follow-on work (answer → article → training → insights)
- •Slack Q&A can become scalable team knowledge
- •Question clustering reveals themes that can guide roadmap decisions
- •Avoiding data silos and keeping knowledge accessible supports the loop
- 32:09 – 45:47
Scaling the approach: sharing tactics, raising technical literacy, and prompting for better answers
They discuss how Al socializes the workflow across the field engineering team through tips and enablement rather than mandates. In lightning round, they cover access concerns, the need for broader technical literacy, and practical prompting like “think harder” plus citing sources.
- •Adoption spreads via internal sharing, not top-down enforcement
- •Engineers benefit when customer-facing teams self-serve answers
- •Everyone needs baseline skills: Git, IDE navigation, reading code
- •Prompting tactics: demand sources, lines of code, and deeper reasoning (‘think hard/harder’)
