Claude Code + 15 repos: how a non-engineer answers every customer question | Al Chen

1. 0:00 – 2:50

   Introduction to Al Chen

   1. ACAl Chen0:00

      The minute I realized I couldn't really do my job was when I was trying to reference our public documentation and trying to provide an answer. It just still wasn't coming up with a answer that my customers were looking for.

   2. CVClaire Vo0:11

      They don't want the docs answer. They want the step-by-step answer of how all these services cascade together.

   3. ACAl Chen0:17

      What I realized is that I can actually pull all of these repos into my VS Code, and I can now use Claude Code to ask our entire codebase questions.

   4. CVClaire Vo0:26

      Did you just say, "Claude Code, write me a script that pulls all these?"

   5. ACAl Chen0:30

      Yeah, yeah. I'm opening up the script right now. It's like, what? Sixteen lines. Didn't have to write this. I just said, "Help me figure out a way to pull the latest main branches into my local repos."

   6. CVClaire Vo0:38

      The reality is we can now all live in a little bit more chaos because the AI navigates all that information for us across systems, right? So you can be in your code querying Confluence. It will find the information. You have to be less precious about where and how you store the information.

   7. ACAl Chen0:55

      Throw it into Confluence, throw it into Notion, throw it into Slack, wherever. That ends up being context you can provide to Claude when you are trying to ask it a question about a customer or about your codebase.

   8. CVClaire Vo1:06

      Let's give Claude Code a little spiff every time it answers a question correctly. You gotta split your quota with Claude Code.

   9. ACAl Chen1:13

      Yeah, it gives you better answers the more bucks you give it or something.

   10. CVClaire Vo1:16

       Coin-operated Claude, that's gonna be my new skill. [upbeat music] Welcome back to How I AI. I'm Claire Vo, product leader and AI obsessive, here on a mission to help you build better with these new tools. Today, we have an episode all about harnessing your code to make your customer's experience way better. Al Chen, who's on the field engineering team at Galileo, shows us how he uses their 15 repositories and Claude Code to answer every nuanced customer question that comes across his desk and use that to make the entire customer base and his entire team a lot happier. Let's get to it. This episode is brought to you by Orkes, the company behind open source Conductor, which powers complex workflows and process orchestration for modern enterprise apps in agentic workflows. Legacy business process automation tools are breaking down. Siloed low-code platforms, outdated process management systems, and disconnected API management tools weren't built for today's AI-powered world. Orkes changes that. With Orkes Conductor, you get a modern orchestration layer that scales with high reliability and brings humans, AI, and systems together in real time. It's not just about tasks. It's about orchestrating everything: APIs, microservices, data pipelines, human-in-the-loop actions, and even autonomous agents. So build, test, and debug complex workflows with ease, all while maintaining enterprise-grade security, compliance, and observability. Orkes, orchestrate the future of work. Learn more and start building at orkes.io.

2. 2:50 – 4:23

   The problem: documentation wasn’t enough

   1. CVClaire Vo2:52

      Al, thanks for joining How I AI. I am really excited about this episode because we've seen a lot about using your code as documentation. You know, we've heard engineers saying, you know, "Docs and code should be in the repo," product managers saying, "Code can now be my documentation for internally facing assets or as I help draft PRDs." But you're gonna show us how you can use code as an asset to create customer-facing things and solve customer-facing problems. So tell me, what, what problem were you facing when you decided, "I'm just gonna clone the repo and fire up Claude Code and solve some of these problems myself"?

   2. ACAl Chen3:33

      Sure. Uh, so, uh, working at Galileo on the field engineering team, I'm on the front lines in terms of working with our enterprise customers, who are typically developers themselves and asking very in-depth technical questions. And the minute I realized I couldn't really do my job was when I was trying to reference our public documentation and trying to provide an answer to, um, my customers, even by u- even using Claude Code or ChatGPT or whatever and trying to take all these different help docs and trying to come up with an answer. It just still wasn't coming up with the answer that the, my customers were looking for. And I just... Background, I'm not a- an engineer. I've never held an engineering role, but I think I know enough to just be dangerous.

   3. CVClaire Vo4:20

      [chuckles]

   4. ACAl Chen4:21

      And I realized

3. 4:23 – 6:03

   Pulling 15 repos into VS Code

   1. ACAl Chen4:23

      that our product, Galileo's product, we're an observability tool for AI, AI applications. If you look at this image here, I'm showing an, like an architecture diagram, high level of all the different services that make up our platform. This is all, like, back end images that you have to, that customers have to deploy onto their Kubernetes cluster. And I realized that all these different, um, services, like UI, API, AuthZ, Comment, they are all individual repos within our Galileo repo. We're not a monorepo. We have, like, multiple different repos. And so what I realized is that I can actually pull all of these repos into my VS Code. Initially, it was just more for me to, like, I want to understand, like, how our code works and how our code is structured. But then when I threw it all into VS Code, which looks like here, you notice along the left-hand side, I'm opening VS Code now, and most of these, uh, directories correspond to one of those services within our architecture. So one repo corresponds with one service. And by having all of these repos in my VS Code, I can now use Claude Code to ask my, our entire codebase questions that are not answerable by our public documentation. And so sometimes I'll get really in-depth questions about, "Well, how does this feature actually work?" And so I'll ask Claude Code, "Look into the API repo, look into the AuthZ repo, and help me come up with an answer. If you can't find the answer, reference other repos within my directory, my cor- my root directory, and help me figure out the answer."And

4. 6:03 – 8:00

   How Claude Code queries the entire codebase

   1. ACAl Chen6:03

      so that's the key unlock was when I figured out I could get way more in-depth, way more technical, and at the same time myself, I can learn quite, learn how our codebase works. And how this has really helped me is I don't have to constantly ping our, you know, team engineering channel with, "Hey, what's the answer to this question? The customer just pinged me about this." And you can imagine engineers being really frustrated when I'm at, trying to, you know, pose these questions, and then the customer asks me a follow-up, then I'm, I'm posting the follow-up in the Slack thread. So I'm sure many of you who are working on the front lines with customers understand how that feels.

   2. CVClaire Vo6:38

      Mm-hmm.

   3. ACAl Chen6:38

      But I've basically reduced all that almost down to zero by pulling all these repos into my local VS Code.

   4. CVClaire Vo6:46

      I'm really empathetic to this, this problem, 'cause I, you know, I used to work at LaunchDarkly leading product and engineering, very technical product. We too had an architecture diagram that looked very similar to that. And again, as a more of a... You know, people think that CPOs, chief product officers, or CTOs are internal facing. No, no, no. We're salespeople. We're always salespeople. You trot us out, and you put us in front of the customer, or you put us in front of the prospect to answer the technical questions. And we had a diagram like that, and I would constantly get these very detailed questions that required very detailed answers, like, "How does your caching work?" And, you know, when you have seven layers of caching in your app, you can give the high level docs answer, but when you're sitting with, you know, an architect in the room or somebody highly technical, they don't want the docs answer. They want the step-by-step answer of how all these services cascade together to build a resilient caching mechanism, for example. And I just think how powerful is it to be in a meeting or in an email back and forth and not just sort of give this high level, but be able to query the current codebase and really understand at a detailed level

5. 8:00 – 8:31

   Why current code beats documentation

   1. CVClaire Vo8:00

      how it works. And I think current is very important because you know and I know this is always evolving over time. So even if you got-

   2. ACAl Chen8:07

      Mm-hmm

   3. CVClaire Vo8:07

      ... the answer right, you know, a month ago, maybe your team shipped an update, or maybe, you know, that method is actually out of date, or the docs are a little bit out of date. And so I do think because the codebase is, you know, at least your main branch is always the source of truth-

   4. ACAl Chen8:24

      Mm-hmm

   5. CVClaire Vo8:24

      ... it becomes a really reliable, uh, you know, uh, context set for you to answer questions

6. 8:31 – 9:54

   The pull script that keeps everything updated

   1. CVClaire Vo8:31

      about how the product operates.

   2. ACAl Chen8:32

      Yeah. And to quickly address your comment about how your code is obviously always evolving, I mean, we're pushing out, you know, multiple features per day, multiple releases, and one thing I've done, wrote this with Claude Code, is I have this script at my root directory that says, like, I just do something called pull all, and they... I'm not sure if this is how other people do it, but it just pulls the main branch into my repo for all the repos in my root directory. So if I do this every day, I kinda get the latest code across, like, all these directories on the left-hand side of my VS Code. So it, the alternative which I was doing before for, like, a few weeks, and I realized this is just asinine that I'm doing this, is doing git pull origin main on every single directory. And it was just, like, not scalable because there's now, like, 15 different repos I have to pull the latest from. So that's kinda how I solved the codebase is always evolving problem to make sure that I'm always getting the most up-to-date information for my customers.

   3. CVClaire Vo9:32

      And I have to ask, did you just say, "Claude Code, write me a script that git pulls all these-"

   4. ACAl Chen9:37

      Oh, yeah

   5. CVClaire Vo9:37

      ... uh, codes" [laughs]

   6. ACAl Chen9:37

      Yeah, yeah. I, I have no idea. This is the s- I'm, I'm opening up the script right now in my VS Code, and it's like, what? 16 lines.

   7. CVClaire Vo9:46

      Yeah.

   8. ACAl Chen9:46

      I haven't... Didn't have to write this. I just said, "Help me figure out a way to pull the latest main branches into my local repos," and it just did it in,

7. 9:54 – 11:40

   Opening projects at the multi-repo level

   1. ACAl Chen9:54

      like, one shot.

   2. CVClaire Vo9:54

      Yeah. The other thing I wanna call out for folks as I'm looking at your screen is I don't think people use this trick enough, which is in VS Code, in Cursor, in whatever your IDE is, um, loading a project at the multi-repo level as opposed to at the individual repo level if you're trying to answer questions across the product is really important. So, you know, there's some, like, context bloat stuff that can come into sort of querying across all those repos and all those files.

   3. ACAl Chen10:26

      Mm-hmm.

   4. CVClaire Vo10:27

      But it would be very painful if you had to go into each of these repos one by one and, like, query and then go into the other one and query. And so I like this idea of opening them all jointly in, in, in your IDE so that when you're querying it with Claude Code or you're querying it, you know, with something like Cursor, um, it can have... It can, it can go across, can traverse across repos and really give you highly contextualized answers.

   5. ACAl Chen10:54

      Yeah. Our codebase just happens to be in multiple repos, but I just pulled them all into this, like, giant Galileo directory here.

   6. CVClaire Vo11:00

      Mm-hmm.

   7. ACAl Chen11:01

      And so everything is, like, at the same parent.

   8. CVClaire Vo11:04

      Yeah.

   9. ACAl Chen11:04

      But if you're in a monorepo, uh, could be... Yeah, actually, I don't know how this would work with a monorepo 'cause I've never done it with monorepo with Claude Code.

   10. CVClaire Vo11:12

       [laughs]

   11. ACAl Chen11:12

       But, um, at least for us at Galileo, this is how, how it works.

   12. CVClaire Vo11:15

       Well, I have, uh, uh, many monorepos, and yeah, you just open it at the right... I would, I would say my advice to folks is open Claude Code or open, you know, your IDE at the right level. And sometimes it's narrow-

   13. ACAl Chen11:29

       Yes

   14. CVClaire Vo11:29

       ... and sometimes you need to go up a directory, and I think really thinking about that. And you could even do that contextualized to the problem you're trying to solve, right? Um, and, and doing that, I think, is really

8. 11:40 – 13:25

   Live demo: answering deployment questions

   1. CVClaire Vo11:40

      helpful. Could you show us just using Claude Code what kind of question you could answer with this code context?

   2. ACAl Chen11:47

      I will, um, give you an example of, um... I guess I'm, uh, I'm a big believer in using shortcuts too, so I use a bunch of custom sh- uh, Claude Code custom commands to help me do stuff. So one thing I do a lot is helping my customers deploy Galileo into their VPC. So I have a, a, a custom command called DPL, which is-It actually references our... The first thing it does is it looks at our Confluence, 'cause we have a whole bunch of Confluence pages about how to deploy into Kubernetes using our different images and stuff like that. So I'll say, "DPL, my customer, um, cannot use CRDs, and they are using Google Secrets Manager and want to deploy the wizard image. Give m-me a step-by-step process on how to do it." This is actually not a super representative, um, query because they're way more detailed than this, and I provide a lot more context, but I want Claude Code to, to focus on, um, looking at Confluence first, 'cause I know that we have a whole bunch of deployment stuff there. And then from there, if it can't find the answer, it will go off into all the different repos along the left-hand side of my Claude Code VS Code to find the answer. So right now it's just using the Atlassian MCP to-

   3. CVClaire Vo13:15

      Mm-hmm

   4. ACAl Chen13:15

      ... pull information from Confluence, and then marries that with, like, our code base to answer a very kind of in-depth, um, deployment question. The one thing,

9. 13:25 – 15:00

   The customer quirks system

   1. ACAl Chen13:25

      um, I'm not sure if we should talk about this now, but w- I started doing, um, this in Confluence, where we have a c- we call it a customer quirks page. These are all kinds of... All of our enterprise customers, you typically have air-gapped environments, so they have all these security measures, and we have to abide by them when we deploy the pro- the product into their, their environment. And so I literally have a page that looks exactly like this, where I have the customer's name at the top level and then a bunch of bullet points with like, you know, here are some things about how they store their secrets. Here's how they do namespaces. You know, here's how they handle sidecars and service-to-service encryption. Things I have no, I know nothing about. But-

   2. CVClaire Vo14:08

      [chuckles]

   3. ACAl Chen14:08

      ... um, as I'm meeting with my customers, I'm putting this all into thi- this one Confluence page. It's a, uh, ever-growing Confluence page. And then this is actually one of the core pages that goes into this, uh, DPL custom command, which is "Look at the customer quirks page. If I'm mentioning a customer that's on that page, look at all their quirks," and then in the response from Claude, it's highly customized, highly tailored to their environment. Because I've seen from working with our DevOps team that y- we can provide a generic answer about Kubernetes or about ClickHouse or about whatever for the customer, but it's like something you could just find online by Googling or using AI. But when it's tailored to specific security requirements and deployment requirements, it, it's way more effective and just gives the customer more trust that we,

10. 15:00 – 17:03

    Living in chaos: why organization matters less now

    1. ACAl Chen15:00

       we know what we're doing, essentially.

    2. CVClaire Vo15:02

       What I love about what you showed here, which is, you know, kind of combining the repository with the Confluence MCP and then both, like, team-generated general documentation as well as you-generated, like, micro-documentation at the customer level, is I've heard so often in my 20 years in enterprise SaaS, like, "What is the source of truth for this information?" Like, I'm sure you've heard this too.

    3. ACAl Chen15:25

       Mm-hmm.

    4. CVClaire Vo15:25

       Like, "What's the source of truth for how XYZ works?" Or, "What's the source of truth for this customer?" And people have spent so much time like, you know, pruning these Confluence gardens and organizing their Slack channels and trying to get people to, you know, get naming conventions right. And, like, the reality is we can now all live in a little bit more chaos because-

    5. ACAl Chen15:45

       Mm-hmm

    6. CVClaire Vo15:45

       ... the AI navigates all that information for us across systems, right? So you can be in your code querying Confluence. It will find... You can kind of point it in the right direction. It will find the information. You have to be less precious about where and how you store the information. Bullet point list of quirks, you know, like-

    7. ACAl Chen16:05

       Mm-hmm

    8. CVClaire Vo16:05

       ... really official docs, whatever, it doesn't matter, because AI is just so much, um, more effective at traversing all that information and pulling it in and making it actionable for you. And I don't think that's anything, like, any human was really proud that they were good at. They're like, "I'm really good at finding, like, the right Confluence doc." Um-

    9. ACAl Chen16:23

       Right

    10. CVClaire Vo16:23

        ... that was never, never the value add.

    11. ACAl Chen16:26

        Yeah. Yeah. I mean, I think, um, even if it's as simple as, hey, you, you came a- come across a really great answer in Slack, like in a really engaging Slack thread, throw that into a Confluence page or save that Slack thread, because I also use the Slack MCP to be able to summarize threads. So if you have, like, just some random, like, just ongoing stream of consciousness of documents you wanna have Claude Code scan, I would just say throw it into Confluence, throw it into Notion, throw it into Slack, wherever, and then that ends up being context you can provide to Claude when you are trying to ask it a question about a customer or about

11. 17:03 – 18:20

    Competing on customer experience, not just product

    1. ACAl Chen17:03

       your code base.

    2. CVClaire Vo17:04

       Well, and the other thing, and this is maybe going back to how I introduced this episode, which is people use AI so much to compete on the field of the product and engineering velocity, and what I mean by that is, like, we're all using Claude Code to ship more product. We're all using AI and Codex to build, you know, better user experiences or more resilient backends or any of that stuff. But there's also a completely different competitive field, which is how you show up in your relationships with your customers. And I think, you know, what you're showing is you can actually use AI to invest and compete on customer experience. And, you know, my hypothesis is when your very complex enterprise customers have you show up, and you don't just say, like, "Here are our general docs to deploy this," and instead you say, "I heard you. I understand what your needs are, and here are your custom docs on how you specifically need to deploy this, and I've already pre-thought about all the problems you've already told me about," you know, just looking, like, in a competitive sense-

    3. ACAl Chen18:09

       Mm-hmm

    4. CVClaire Vo18:10

       ... that's gotta come across as a much more, um, enjoyable customer experience on the receiving end, and allows you to position yourself not just as a great product, but as

12. 18:20 – 20:05

    Should customers be able to query the code directly?

    1. CVClaire Vo18:20

       a great team that's gonna service your customers well.

    2. ACAl Chen18:22

       Yeah, I, I hope so. I mean, I think our customers... I think my customers are hopefully enjoying the answers I provide and the in-depthness that I provide. I think I've thought about taking this to the extreme, which is there-- we have certain, I have certain customers who are, like, you know, very in the weeds. They wanna know things, like, right at this very second, and I'm literally taking their question and then just, like, saying, "My customer then asked me this," because they can't see your code, but me, Al, I can see the code. Help me get the answer. And so if I take that to the logical con- logical conclusion, it's like, why can't we just share our repos with the customer? 'Cause then they can just start querying our repos directly to get the answers they need instead of me as kind of like the, quote-unquote, "middleman." And, you know, the issue is that, like, our code is proprietary and all that kind of stuff, but I have seen, um... There's actually a case study from LangChain, and since a lot of LangChain's, um, repos are, you know, it's open source-

    3. CVClaire Vo19:19

       Mm-hmm

    4. ACAl Chen19:19

       ... like, their support agent bot actually does a lot of things I do, but it is able to query, you know, all the public open source repos. And any of you out there who are trying to use LangChain or LangGraph, you can just pull all those repos down to your local machine and then ask questions, of course, using Claude Code or Cursor or whatever. Um, but I've ha- kind of went through that kind of thought experiment of, like, I'm still kind of a bottleneck in terms of answering my customers' questions 'cause I kind of, like, hold the keys to our code. But if they somehow had a, had a sanitized version of it, then maybe they could just self-answer their questions too, because they're also all using VS Code and Cursor and Claude too, but they just don't happen to have our, you know, proprietary

13. 20:05 – 25:46

    Where humans still add value

    1. ACAl Chen20:05

       codebase.

    2. CVClaire Vo20:06

       Yeah. I was, I was gonna ask you, are you worried that, like, the Al bot is coming and you're, you're cut, you're cut out of it? And I, I'm, I'm just curious how you think about then when, like, you, you... Again, like, the highest order of you is not to be a pass-through, and I don't think you think of that yourself as that. And so where does the human in these relationships powered by AI, you know, add, add the value?

    3. ACAl Chen20:31

       Well, to... I don't just blindly copy and paste the answers I get from Claude Code to my customers in Slack or an email or wherever. I still try to proofread everything, and I actually do, like, try to make it sound more human. And you can then say the argument, "Oh, why don't you just use Claude Code to make your answer sound more human?" And I think all of us know when we get an answer that's from an AI-

    4. CVClaire Vo20:52

       Mm-hmm

    5. ACAl Chen20:52

       ... and it's, you know, things like, you know, you'll see, like, a bullet point saying, like, "In summary, here are the things you need to do to make sure your ClickHouse works within..." So it's like removing things like that that just make it seem like it's from a bot just makes it seem more human.

    6. CVClaire Vo21:08

       Mm-hmm.

    7. ACAl Chen21:09

       And we've actually... I mean, this is kind of going into, going behind the scenes of how we work, but we've been dinged sometimes where the customer will say, "Can you just not give me an error response and just give me, like, a human proofread of it and tell me how it applies to me?" Because typically the response is way too verbose. It has way too much information, and the customer just wants to know, "Give me, like, the bottom line up front. What do I need to know to, like, deploy this image onto my cluster?" And so that's where the human... I, I still com- see myself as a human providing value and culling that down to what they actually need. And I would say even for some of the more in-depth technical questions, I still try to get an engineer's perspective on it to make sure, like, Claude Code is not hallucinating or not saying anything out of nor- out of the ordinary. Even in my system prompt, I always... You know, in my Claude Code, I say things like, "Don't make anything up. Always cite your sources. Point me to the line of code where you're getting this information from." But even with that, if I don't fully understand how this function works or whatever, I'm still pinging, like, the engineering channel to say, "Hey, this is what Claude Code told me. Does that jive with what you're thinking?" And there are times when I'm wrong or Claude Code is wrong because our engineers have been thinking about refactoring into this new model which is not captured in our codebase anywhere. It's just captured in, like, a meeting note somewhere or just, like, you know, hallway conversations. And so those are the things that I'll never be able to query, let's say, in, in Claude.

    8. CVClaire Vo22:45

       Yeah. I would, I would say the other thing that, you know, where I see humans adding value, and I say this all the time, which is, like, Riz is the only moat, which is at some point, you know, people just want to have a face and a trusted personal relationship, you know, with the folks. And this is, like, m- my enterprise showing. But, like, with the folks that are selling them software, you wanna know that you have somebody to call. You wanna know that you have somebody that can gather the right f- the right folks around your team and your deployment, and, you know, you wanna enjoy working with that person. And I will just say I get a lot of, um... It is very fun for me to build with these tools, with AI tools, but I wouldn't say my AI colleagues are, like, the most fun to hang out with, which is like, I'm not, like, always looking forward to, like, my, my Claude Code session. Like, I wanna really chitchat with, with good old Claude. Um, and I do think you still have that relationship with, you know, your human partners, your human colleagues, all that sort of stuff. And so I think there is a piece of that that's just not gonna get cut out. And honestly, I gave this talk, uh, I don't know, two years ago. I said, "PM is dead." And people were like, "Well, what else should we do?" And I was like, "Get into sales." Like, that's not going away. Customer-facing stuff is not, is not going away. Um, so for anybody that wants to survive, you know, the, the incoming apocalypse, I do think customer-facing roles and spending more time customer-facing is a really important part of-

    9. ACAl Chen24:08

       Oh-

    10. CVClaire Vo24:09

        ... everyone's job.

    11. ACAl Chen24:10

        Absolutely. If you're working enterprise sales, like, that is all people, handshakes, um, lunches, dinners. So that will never be replaced, I think-

    12. CVClaire Vo24:20

        No

    13. ACAl Chen24:20

        ... by AI anytime soon.

    14. CVClaire Vo24:22

        Well, you know, and there might be a generational shift though here, I think, as, as we sell, as we sell. We'll see. You know, I used to say my, my joke in enterprise sales and the biggest, um-The, the biggest headwind to enterprise sales was I was starting to sell to millennials who like wanted you to text when you showed up at their door. They didn't want you to knock on their door like there was a subscription.

    15. ACAl Chen24:44

        Oh, yeah, yeah.

    16. CVClaire Vo24:45

        [laughs] Um, we'll see. We'll see how enterprise sales changes generationally.

    17. SPSpeaker24:49

        [upbeat music] This episode is brought to you by Tines, the intelligent workflow platform powering the world's most important work. Business moves faster than the systems meant to support it. Teams are stuck with repetitive tasks, scattered tools, and hard-to-reach data. AI has huge promise, but struggles when everything underneath is fragmented. Tines fixes that. It unifies your tools, data, and processes in one secure, flexible platform, blending agentic AI, automation, and human-led intervention. Teams get their time back, workflows run smarter, and AI actually delivers real value. Customers now automate over 1.5 billion actions every week. Tines is trusted by companies like Canva, Coinbase, Databricks, GitLab, Mars, and Reddit. Try Tines at tines.com/howIAI.

14. 25:46 – 29:16

    Using AI for reactive Slack support

    1. CVClaire Vo25:47

       All right. So we have, just to recap, we've shown how you use all these repositories in your very complex codebase. Pair that with Claude Code, um, which is made more efficient through a couple like shortcuts and scripts to be able for you to answer customer queries and then also build custom deployment plans for your customers anchored in exactly how your code works and exactly how their infrastructure works, making everybody happier and getting customers off the ground quicker. But-

    2. ACAl Chen26:20

       Mm-hmm

    3. CVClaire Vo26:21

       ... there are also instances where you need to be doing more reactive, um, support in, in different channels, and I know you're using AI for that. So you wanna walk us through how you're using AI in Slack and supporting customers there?

    4. ACAl Chen26:33

       Yeah. So like many, I say, digital AI native companies, we do a lot of our customer support through Slack. You know, we have external channels with our customers, and not... I, I mean, I come from the w- I used to work in a world where everything was through like a central Zendesk or Intercom or whatever. But for enterprise customers, it's kind of like a on-the-go, always kind of on kind of thing. And so we use a tool internally called Pylon for monitoring all our different external Slack channels, and I'm gonna show you what this looks like in this tab. And this is an example of a conversation I had with a, a customer asking in-depth quest- uh, questions about, like, our Galileo callback function and how it emits different events. And as you can imagine, I was using Claude Code to help answer these questions in addition to using our docs. But when you're looking at a conversation like this in Pylon or in Slack, the first thing you have to think about is like, I wonder if, like, I could turn this into a help article, or if I should update our docs, or will other customers benefit from the knowledge that's being trapped in this little Slack thread? And so what Pylon allows us to do is looking at a really long Slack thread, it can help you generate a help article. And right here, I already have one that's associated with this specific conversation, but it's literally just clicking on Add Article, Generate A- Article Draft, and then we have these different templates, and it just creates, like, this article for you on the fly. Now, this is not rocket science. You could copy and paste the whole Slack thread, put it into any AI tool you want to generate an ar- help article. The main thing with Pylons is everything's kind of just, like, in one interface, so you don't have to worry about, like, copy and pasting and putting links together. So this is kind of like that draft that this came up with. And then we have this ongoing list of articles based on real customer conversations, and those articles are abstracted to not show any specific customer information. But then when we publish these articles, they go, they go into this knowledge base, which is also public knowledge base, and this is kind of like the living truth of, like, in-depth, in-the-weeds, uh, co-con- questions about deployment, about how Galileo works, and it's always way more in depth and way more up-to-date compared to our docs. Because our official docs require pulling down the docs repo, submitting a PR, getting it approved, so on and so forth, and so it's a lot more of a polished process. Whereas with these knowledge base articles, it's kind of like just on the fly, you have a Slack thread you want to summarize, use it, uh, create it in Pylon, and then it just automatically gets auto-published

15. 29:16 – 32:07

    The “and then” workflow discovery

    1. ACAl Chen29:16

       to this, uh, knowledge base site.

    2. CVClaire Vo29:18

       So one of the things that I love about this is this represents my, the, like, what I call the "and then" workflow discovery in AI. Which is, I say, imagine you had an infinitely staffed team, and you were faced with a task, and every time they did one step of the task, you asked, "And then?" And they were able to do it. So it's like-

    3. ACAl Chen29:37

       Mm-hmm

    4. CVClaire Vo29:37

       ... I got a Slack query from a customer, so I answered it, and it was like, "If you had a perfectly staffed team, what would you do next?" And it'd be like, "And then I would turn that into an article." And it was like, "Okay, and you turn it into article, and then what you do?" It's like, "And then I would share that with our customer success team and train them on this answer because, you know, everybody needs to know this information." It'd be like, "And then?" And you'd be like, "And then we could probably do, like, long-tail SEO off all these questions."

    5. ACAl Chen30:04

       [laughs]

    6. CVClaire Vo30:04

       And I think you can, like, chain, chain these, like, "and then," um, you know, workflows to actually build out, like, a pretty cool, you know, virtuous cycle system based off a single action. And because, again, like, the cost of doing any one of those collapses to zero, you can really pull the thread of these tasks that, like, no human team would have the capacity to really do. But if you think of it as a system, it helps your human teammates, it helps your customers, and you can get a lot of stuff done. We have a couple episodesUm, Matt at Suzy showed kind of a, a version of this where he takes a customer... a recorded customer call-

    7. ACAl Chen30:45

       Mm-hmm

    8. CVClaire Vo30:45

       ... and, like, is bidding on AdWords for, like, phrases the customer says, and, like, spinning up blog posts and doing, like, sales coaching off of it. So I think this is, like, a very similar example, which is you have this, like, you know, atomic unit of a question in Slack, and you've turned it into something that benefits, benefits the full team.

    9. ACAl Chen31:04

       Yeah. I think if you go back to pre-AI days, and I remember doing this with Intercom, was you... we wanted to see what are our, what are our users talking about the most when they're asking us questions.

    10. CVClaire Vo31:13

        Mm-hmm.

    11. ACAl Chen31:14

        And so if you start clustering all these user questions and insights into different themes and categories, those can end up determining, determining your product roadmap, too.

    12. CVClaire Vo31:26

        Yep.

    13. ACAl Chen31:26

        And so I think with AI, just kind of automates mu- a little bit more of that without you having to, like, do the manual sorting, grouping within, like, Google Sheets or whatever. I know there's, like, platforms you can buy that do this for you. I, I think there's one called, um, Interpret-

    14. CVClaire Vo31:42

        Mm-hmm

    15. ACAl Chen31:42

        ... which I've used in the past. Um-

    16. CVClaire Vo31:44

        There, there, there's... They've been an... a How I AI sponsor, so thank you, Interpret. [laughs]

    17. ACAl Chen31:47

        Yeah, yeah. So, um, but, you know, I think, again, depending on how you wanna view, view this whole virtuous life cycle, maybe you don't want all of your data to be, like, in a silo in one place-

    18. CVClaire Vo31:59

        Yep

    19. ACAl Chen31:59

        ... and you want it to be more open. So there's that to com- to think about, too. But yeah, AI definitely helps, to your point, make that virtuous cycle n- for customers,

16. 32:07 – 34:07

    Scaling processes across the team

    1. ACAl Chen32:07

       but also for your product team.

    2. CVClaire Vo32:09

       So I have a question. Um, is this the, the Al system, or is this the Galileo, you know, field engineering system? Which is, you know, you have this great workflow. You've discovered all these things. How does this sort of process get scaled out, shared, taught throughout the organization so that everybody that interacts with customers is benefiting from all the tips and tricks that you're figuring out yourself?

    3. ACAl Chen32:32

       Sure. So I, my previous background was I've worked in kind of the no-code, low-code space, and I'm a big believer in systems, tools, processes, and the tools that help you create those things. And so when it comes to, is this the Al way of doing things? Yes, it's my way, but I'm also very, probably one of the mo- more opinionated people on the field engineering team about, like, how we should be doing things in terms of talking to customers, answering their questions, and pulling in the right context. And so I've told multiple people, like, "Pull all the repos into your local machine and have Claude Code run an init command to index the whole code base or whatever." And I'm just, like, constantly sharing these tips and tricks to my teammates to make sure they're also, um, functioning at their capacity. So it's my way, but I'm... I would say I'm also very opinionated about how we should do things, 'cause I've done things the hard way, the manual way, and this way to me is, like, just 10 times way more productive. So, um, we don't have, like, a specific, like, "Oh, because Al's doing it now, the whole team has to do it." It's more just, like, people show, "Here's the problem I had, here's the results I had with Claude Code, whatever, and here's why I think you should adopt my, um, solution." And I'm constantly having that conversation internally about, like, how do we break out of certain processes that I think are slowing us down, and, um, how AI can be in- infused into all those processes as well.

    4. CVClaire Vo34:00

       Well, and now you're sharing to all of our How I AI audience on how they can do that.

    5. ACAl Chen34:04

       Yeah, there you go.

    6. CVClaire Vo34:04

       So you're having more impact than just

17. 34:07 – 45:45

    Lightning round and final thoughts

    1. CVClaire Vo34:07

       on your team. All right. Well, so to just recap again, your code is your source of truth. It can help you answer customer questions. It can help you document, um, customer solutions. You can also do that with other channels like Slack, and then, like, create these virtuous loops of solving a single customer's problem, and then a system to solve that problem more scalably across your entire customer base for yourself and for your teammates. Um, very, very high impact episode. I think people are gonna have a lot of takeaways from this one. Super practical for all my friends that are customer facing out there on things they can do starting tomorrow to use AI, um, to, to give their c-customers a better experience. Let's jump into lightning round questions, and I have one that's really top of mind.

    2. ACAl Chen34:52

       Mm-hmm.

    3. CVClaire Vo34:52

       Which is it seems like you have a very healthy culture at Galileo, but I can imagine teams, especially engineering teams, that are like, "Oh, no, no, no, no. I, I don't really want the customer-facing folks, like, going into our repo, querying it, and then just YOLOing answers over to our customer base," especially in a more technical product that really requires deep technical understanding. I think you've proven that there's a lot of value in doing that, but what would you say to those teams that are a little bit more hesitant about ungating access to the repo to non-technical roles?

    4. ACAl Chen35:29

       Um, I think from the engineering, engineer's perspective, I would look at it as how many... I would try to think about how many times in the last week, in the last day, have you been asked a last-minute question on Slack, a last-minute DM, ping, mentioned in a thread where how does this thing work? Where is this... How do we make sure that this is functioning the way it should be? And you're constantly the source of... You're the bottleneck for answering that question. And if you provide a system, kind of similar to what I have, to your, uh, your customer-facing team, then you kind of just take away that toil and the constant, like, on-callness of, like, answering these random product and engineering questions that is already in your code base, or maybe it's already living in your Confluence or something like that. So I think that's really the biggest takeaway for me is how much of your time is being sucked away from your customer's team because they don't have access to the code. And I mean, I think there's some no code... I mean, I mean, I think you can maybe pull in your code into Clo- Cla- Claude Cowork, which is a little more no-codey and other kind of, like, more no-codey ways of doing things. But I think-What I've shown is, I think, the most performant way of being able to pull your code and get answers out. Um, so I think that's kind of, um, from an engineer's perspective, how much time can you save, and then also how more effective your, your customer, um, facing org can be. And I think the corolar- corollary to that is that our field engineering team is very technical, and so maybe you increase the hiring bar for your customer success or customer engineering team to feel comfortable using GitHub and pulling repos into your local machine. And so that could be... Today, if it's, if they're not technical, just doing, like, a simple tutorial or enablement session on how do you use GitHub, how do you use Git commands, um, things like that. And there might be some self-learning you have to do on the side, too. But I think once you're, once you have your environment set up... That's always, like, the hardest part about this whole exercise, is getting your environment set up. Once that thing is set up, then using Claude Code is just like using any other AI chatbot. Um, so I think there's, like, a few different ways I'd approach it from, to democratize access to your repos.

    5. CVClaire Vo37:52

       One of the things I was gonna say is I often tell people this is the era of the hard skill, which is, no matter what role you're in, sorry babe, you gotta, like, learn a little bit how to code. You have to learn a little bit what Git works like. You have to be-

    6. ACAl Chen38:06

       Mm-hmm

    7. CVClaire Vo38:06

       ... okay opening up some code you don't understand in an IDE, because that's just gonna be the substrate by which we communicate for the next three years. It's gonna go, like, closer and closer to the code-

    8. ACAl Chen38:18

       Yeah

    9. CVClaire Vo38:18

       ... because these LLMs are ex- extremely good at understanding code. And so I think across the board, people just need to become more technical and develop hard skills around code, even if your job is not code. I think the second thing that I tell people is, um, it, there's no better time to learn how to code.

    10. ACAl Chen38:39

        Mm-hmm.

    11. CVClaire Vo38:39

        Truly no better time to learn how to actually code. And I think people that are shipping with Claude Code, but not using that as an excuse or a support to learn some fundamental software engineering concepts are missing 50% of the value. Like, I had to teach myself how to code out of a book. Like, literally out of a book. It was... I had a book open, and then I would look at my single screen, because none of us had two screens. That would be crazy. And I would, like, read the book, and I would type the book in the wor- like, the words in the book in code, and would press Enter, and it would say, "Hello, world." And that was my life. And now you have this, like, magic, super patient, infinitely wise, you know, like, teacher in your computer that you can use to learn to code, and I think... You know, you talked a little bit about Kubernetes and how you, you scaled up on that. So I'm curious your thoughts on just upleveling technical skills using some of these, these tools.

    12. ACAl Chen39:38

        I, I think the meta takeaway is, like, you just have to be curious about, like-

    13. CVClaire Vo39:42

        Yeah

    14. ACAl Chen39:42

        ... how things work. I can't really say anything else, else besides that. It's kind of like, I come from that same world, too, of, like, looking at a book, and then I would say the graduation above that was knowing how to write a good Google query.

    15. CVClaire Vo39:55

        Yeah.

    16. ACAl Chen39:55

        And then-

    17. CVClaire Vo39:56

        Stack Overflow. [chuckles]

    18. ACAl Chen39:56

        ... going to Stack Overflow, and then how many of you resonate with this, where you go to some Stack Overflow Q&A, you copy and paste the code. It doesn't work perfectly, of course, so you're Googling the m- the error you get from that thing you copied and pasted. And of course, everyone in Stack Overflow is super snarky, and it's, like, not a kind of healthy conversation. And then to your point, you have this, like, infinitely patient, infinitely kind assistant who never gives you the wrong code snippet from Stack Overflow. It's always, like, tailored to... Back to everything I'm saying, it's tailored to your needs, to what you want. And then if you go the extra step, and like what you said, and then what? If you go the extra step and say, "Okay, thanks Claude, you told me this is the answer. Tell me why this works." And then it's, you start getting into Kubernetes and into the deeper in the, in the weeds things. But of course, you're not gonna know everything right off the bat, so you can say things like, "Well, explain to me in simple terms. Explain it to me like I'm five." And so you're just kind of pulling on that thread, and I sometimes do get lost going down the rabbit hole, um, but I've never found a situation where not going down that rabbit hole does not help me in my day-to-day job, especially in AI, where everything's, are, is, is moving so fast.

    19. CVClaire Vo41:15

        Yeah. And this is just to, just to make everybody feel comfortable, this is not just a beginner thing. And I find myself doing this with, um, GPT-54, which is, like, a powerhouse model, and also, like, talking to the most esoteric senior software engineer you've ever met, where it, like, explains its plans in these very technical terms, and I'm like, "Dude, just, like, explain to me what you're doing in number one."

    20. ACAl Chen41:41

        Yeah.

    21. CVClaire Vo41:41

        "Tell me in plain language. I do not need the technical details. Like, just tell me in plain language." And again, it comes from this curiosity of I wanna make sure I understand the fundamental concepts of what you're talking about, and I wanna make sure I'm learning both my codebase and general principles as we go. And so I do think that curiosity mindset, no matter what your seniority level is, your experience with technology, you can always learn, learn something better. Okay, my last question before we get you out of here, um, when AI is not giving you the right answer, it's giving you AI slop that it, it wants to email to your customer, what is your prompting technique?

    22. ACAl Chen42:19

        I, I'll, I'll say one thing first off the bat is, like, I'm very relentless when it comes to getting the right answer from Claude or from AI. Like, I treat it like my entry level analyst, you know? To that I can just, like, throw a billion questions at. I... 'Cause I used to be an analyst, and, uh, I come from the world where, like, you were just expected to crank. And so I'm relentless when it comes to asking AI to do things for me, especially when it comes to answering customer questions. I think-The one prompt strategy I use is like, I mean, you've probably heard versions of this before, which is like, you know, my customer will, you know, churn if I don't get this right, or [laughs] you know, like, um, my, my quota is dependent on getting this, this thing done. Um, so those are kind of ways I've approached it, but th-those are, like, half answers. I think the, the real answer, and this goes back to curiosity thing, is like, mm, think hard. Think harder about why you're giving me this answer. Think hard about why this is right. And so in Claude Code, there's actually, like, this think hard, think harder paradigm of, like, how much reasoning it does to come to the answer. And so it's just going, going one step deeper and saying, like, "You gave me the answer. Tell me why you think this is the an- why this is the right answer, and give me the sources for what provided you with this reasoning." Um, so I think going that one extra step, especially for those questions where you're, like, not quite sure if it's the right answer, and, like, you're reading the code, and it kinda makes sense, but just going that one extra query to make sure you're getting the right response will, um, sometimes, you know, give you new insights about your codebase that, um, your product that you haven't thought about before.

    23. CVClaire Vo43:59

        Okay. I like the practical, like, force the enhanced reasoning, think hard, think harder.

    24. ACAl Chen44:04

        Yep.

    25. CVClaire Vo44:05

        I don't want people to miss... You tell people you're gonna miss quo- I mean, like, let's give Claude Code a little spiff every time it answers a question correctly. You gotta split your quota with Claude Code. That's really what we need to do.

    26. ACAl Chen44:19

        [laughs]

    27. CVClaire Vo44:19

        Um, and say, "Look, I'll give you a point on this deal if we can answer this ques- [laughs] this question well."

    28. ACAl Chen44:26

        [laughs]

    29. CVClaire Vo44:26

        Very, very funny. And I think today, um, it'll be live by the time this episode goes live, Stripe just released this, like, payments protocol so you can pay your agents. So you can, you know, toss it a couple agent bucks or whatever, uh, every time that-

    30. ACAl Chen44:39

        Yeah, it gives you better answers the more bucks you give it or something.

Episode duration: 45:47