How to digest 36 weekly podcasts without spending 36 hours listening | Tomasz Tunguz

1. 0:00 – 3:32

   Introduction to Tomasz Tunguz

   1. TTTomasz Tunguz0:00

      I have a list of 36 podcasts, but I don't have 36 hours every week to listen to 36 podcasts. So what I did is I created a system that goes through each of those podcasts every day and downloads the podcast files, and then transcribes them.

   2. CVClaire Vo0:14

      Can you show us how it's actually built? Like, where do you get this feed? It sounds like you run it locally. How does this all work?

   3. TTTomasz Tunguz0:20

      I wrote this thing called the Parakeet Podcast Processor, and this podcast processor basically takes in a file, and what it'll do is it will read the file, it'll download it, and then it will convert it via FFmpeg. Then that will take the audio and convert it to text. So here's the podcast summaries for today. There's Lenny's podcast, the host, the guests, a comprehensive summary. So here's a conversation with Bob Baxley, key topics, and then key themes. The part that's most invaluable for me are these quotes, and those quotes, I'll read them. It'll suggest a bunch of actionable investment theses for a venture capital firm, which is put into the prompt, like, "Okay, maybe we should be looking at AI-assisted design tools."

   4. CVClaire Vo1:00

      You've gotten not only the content you want, but the user experience you want. You control it end to end, and you can build this hyper-personalized software experience. [upbeat music] Welcome back to How I AI. I'm Claire Vo, product leader and AI obsessive, here on a mission to help you build better with these new tools. Today, I have Tomasz Tunguz, a legend in the enterprise software business and founder of Theory Ventures, which invests in early-stage enterprise AI, data, and blockchain companies. Tom is followed by over a half a million folks on his blog and LinkedIn, and he's gonna show us today how he uses AI to keep up with all the podcasts, including this one, and draft blog posts that would be approved by your AP English teacher. Let's get to it. This episode is brought to you by Notion. Notion is now your do-everything AI tool for work. With new AI meeting notes, enterprise search, and research mode, everyone on your team gets a note-taker, researcher, doc drafter, brainstormer. Your new AI team is here, right where your team already works. I've been a longtime Notion user, and have been using the new Notion AI features for the last few weeks. I can't imagine working without them. AI meeting notes are a game changer. The summaries are accurate, and extracting action items is super useful. For stand-ups, team meetings, one-on-ones, customer interviews, and, yes, podcast prep, Notion's AI meeting notes are now an essential part of my team's workflow. The fastest-growing companies, like OpenAI, Ramp, Vercel, and Cursor, all use Notion to get more done. Try all of Notion's new AI features for free by signing up with your work email at notion.com/howiai. To celebrate twenty-five thousand YouTube followers on How I AI, we're doing a giveaway. You can win a free year to my favorite AI products, including v0, Replit, Lovable, Bolt, Cursor, and of course, ChatPRD, by leaving a rating and review on your favorite podcast app and subscribing to YouTube. To enter, simply go to howiaipod.com/giveaway, read the rules, and leave us a review and subscribe. Enter by the end of August, and we will announce our winners in September.

2. 3:32 – 5:06

   Overview of the podcast ripper system and its components

   1. CVClaire Vo3:32

      Thanks for listening. Okay, Tom, I'm so happy to have you here, because you are gonna show us how you are solving a problem I'm creating for you.

   2. TTTomasz Tunguz3:41

      [laughing]

   3. CVClaire Vo3:41

      And the problem, the problem I'm creating for you is I am creating yet another piece of interesting content that you have no time to consume, certainly [chuckles] the format that we get it out. And I know to you, content is a really interesting source of ideas, of trends, of companies. So tell us what you built and why.

   4. TTTomasz Tunguz4:05

      Absolutely. Well, thanks for having me on. So I, I don't... I prefer to read than to listen, uh, because I can skip ahead, and I think there's a lot of information inside of podcasts that people share that I would love to know, and so I built, I guess, what I call a podcast ripper. And the idea is, I have a list of 36 podcasts, this one included, that I really admire and I want, I want to learn from, but I don't have 36 hours every week to listen to 36 podcasts, right? So what I did is I created a system that goes through each of those podcasts every day and downloads the podcast u- uh, files, and then transcribes them using, initially, it was open source, or OpenAI's open source, Whisper, which takes audio and converts it to text, and then there's a new version called Parakeet, which Nvidia released, that runs really well on a Mac. And so I'll take that text, and then I'll run it through a prompt, and it will spit out a whole bunch of different things. Uh, it'll spit out a high-level summary or

3. 5:06 – 6:59

   Demonstration of the transcript cleaning process

   1. TTTomasz Tunguz5:06

      whatever I ask it to.

   2. CVClaire Vo5:07

      Okay, can you show us how it's actually built? Like, where do you get this feed? Do you... It sounds like you run it locally. How does this all work?

   3. TTTomasz Tunguz5:14

      So I initially downloaded the Whisper app, and I, what I did is I wrote this thing called the Parakeet Podcast Processor, and this podcast processor basically takes in a file, and what it'll do is it will read the file, it'll download it, and then it'll convert it via FFmpeg, which is a library that converts different kinds of files. Then that will take the audio and convert it to text. And then I use, uh, Gemma 3, which is really good at this, to actually clean up the transcript. So if we search for, uh, the Ollama model... Basically, what I'm doing is I'm just cleaning up the file here. "You're a transcript editor. Clean up this podcast while preserving all the content. Keep the same length."... remove the uhms and the ahs, preserve all technical conversations, and that returns a clean transcript. And so on a given day, there might be five or six different transcripts that need to be transcribed. And then what I'll do is, it runs through the Parakeet Podcast Orchestrator, or actually it's just the Podcast Orchestrator, which is here. And so I'm storing each of the files that I'm transcribing in a local DuckDB, which is a little database that says, "I processed this particular podcast on this particular day." And then I save the transcripts, and I take all the transcripts on that particular day from the database, which is here, and then I send them through a prompt, which... Let's see if we can find it. Summarizes, here, the daily summarizer. So it generates a daily summary document, um, which is here.

4. 6:59 – 10:20

   Extracting quotes, investment theses, and company mentions

   1. TTTomasz Tunguz6:59

      It'll produce a file that looks like this. So here's the podcast summaries for today, uh, June the 13th. So there's Lenny's podcast, the host, the guests, a comprehensive summary. So here's a conversation with Bob Baxley. Uh, key topics, so here he's talking about his philosophy and company culture, and then key themes. And the, the key- the, the part that's most valuable for me are these quotes, and those quotes are then... You know, I, I'll read them. It'll suggest a bunch of actionable investment theses for a venture capital firm, which is put into the prompt. Like, okay, maybe you should be- we should be looking at AI-assisted design tools. And then that might kick off a market map. We're really thesis-driven, so maybe that starts a conversation on a Monday, and we decide to staff a market map. Then it'll produce these noteworthy observations, which are, uh, actually put into tweets. So here are the Twitter post suggestions. So I haven't done this yet, I'm still working on the prompt, but the idea is, like, could we actually automate, like, linking back to people who we really like? And then another part, this is a little out of order, but then another part here is, are there startups that are mentioned within these podcasts that we should know, right? So here's Airbnb, Google, Amazon, Stripe. We know all [chuckles] these guys. I don't know what this company is, and so this might go into our CRM, right, to be enriched. And, and then the last is, we'll actually generate prompts for blog posts in the style that I write. And then this will go into a Python pipeline to actually machine-generate, uh, blog posts.

   2. CVClaire Vo8:32

      So before, [chuckles] before we get to the, the, um, machine-automated AI blog post pipeline, I have a couple of questions about this process-

   3. TTTomasz Tunguz8:40

      Mm-hmm

   4. CVClaire Vo8:40

      ... 'cause I think you did a couple interesting things. One, I have a question, i- if you found higher quality by cleaning up the transcripts, like, how much did that incremental input quality piece actually help your, your output?

   5. TTTomasz Tunguz8:57

      So it helped... So initially, I was trying to get-- the answer was initially a lot, and then over time, less. Because initially, what I was trying to do was to find these companies, I was using named entity extraction algorithms from Stanford, or the Python Library, and, uh, it was having a really hard time. Uh, and so I was cleaning it up, cleaning it up to try to get the performance to improve, and then I just pushed it [chuckles] to a really large lark language model, and it spit it out much better. And so the cleaning is not that useful anymore.

   6. CVClaire Vo9:29

      Yeah, I was looking- 'cause I was looking at it, and you're focusing on, like, proper nouns, company names, and so I'm assuming if you want to extract something like Stripe, which has many, [chuckles] many meanings, um, getting it into a proper noun format, for example, would help with that extraction. But you're saying, as you could just use, as opposed to these kind of package libraries for specific machine learning use cases, instead just send it to an LLM, that ended up just meaning you could worry less about the input quality of, of your transcripts and more about the kind of prompting and structure here of the output.

   7. TTTomasz Tunguz10:02

      Yeah, that's exactly right. So my goal initially was to do everything locally, and so I was using Ollama, I was using that Stanford library, Parakeet is run locally.

   8. CVClaire Vo10:12

      Mm-hmm.

   9. TTTomasz Tunguz10:12

      And then what I realized is, particularly for the named entity extraction, more powerful machines are much better.

   10. CVClaire Vo10:19

       Yeah.

5. 10:20 – 12:38

   Why Tomasz prefers terminal-based tools

   1. CVClaire Vo10:20

      And so- and then I have to ask another question, which is, everybody's gonna look at this, and they're gonna go, "What the hell is he typing in?" [chuckles] Like, we have a couple people that are like, "Why in the terminal?" So I'm just curious, you know, did you ever think about putting a UI on top of this? Do you just... You seem very comfortable in the terminal, so it seems to work for you. I'm just curious about where you decided to focus your, uh, user experience efforts on this personal-

   2. TTTomasz Tunguz10:43

      [chuckles] Well, okay, so I love the terminal. I read this blog post by Dan Luu, with two Us, where he was talking about latency, and the latency between, like, the keyboard and the computer, and it turns out that the terminal is actually the application with the lowest latency. And the lower the latency, the less frustration you have using a computer. So during COVID, I decided to learn how to use a terminal, and since then I've sort of lived in it. And so, like, my email client is a terminal-based email client, and I u- I use that because it's really fast, and then I can also script different things, so I can delete 10 messages at once, or I can call an AI to actually automatically respond to an email, uh, or add a company to a CRM. So that was really important. But at a high level, like, I think it's, um... I've just become really comfortable with it. It's really fast. And then the last thing I'll say is, I think Claude Code is an amazing product, and the great part about what Claude Code does is, I have about 2,000 blog posts. I can just go into Claude Code and say, "Modify the files in this way," or, "Change the blog post theme," or recently I launched a blog post generator, which takes all of the content that I have on the blog, and you can ask it a question, and it will write a blog post for you about your particular question. And I did that all using Claude Code.

   3. CVClaire Vo11:59

      Yeah, I mean, I, I have two sort of thematic things that I think of while observing this, this workflow and your love for the terminal. I agree, Claude Clode- Claude Code is an amazing product, and it's a really well-designed terminal-based product. I love it. I love that-... you have this constrained surface area in which to, like, communicate progress and latency and changes, and I think it's really thoughtfully designed. So for anybody out there building dev tools in particular, learn how to design in the terminal. [chuckles] It's so, so important, 'cause you can make really fabulous products for, I guess, people like you and me that say things like, "I picked up the terminal over COVID as my, [chuckles] as my hobby."

6. 12:38 – 15:31

   The benefits of personalized software versus off-the-shelf solutions

   1. CVClaire Vo12:38

      The second thing that I was thinking about is since generative AI has become mainstream, every single person has said, "Somebody make a podcast digest application." Every single person I know is like... It was one of the first projects I made. I made my kids a podcast digest-

   2. TTTomasz Tunguz12:55

      Oh, lovely

   3. CVClaire Vo12:55

      ... their favorite podcast, and it made little, um, quizzes about the topics that they could answer. Super cute. So I think it was a very common use case. But what I was thinking is, no startup is gonna be like, "You know what's gonna be a huge TAM company? A terminal-based podcast transcript processor and thematic extraction generation engine."

   4. TTTomasz Tunguz13:19

      [chuckles]

   5. CVClaire Vo13:19

      And I, [chuckles] I think this is such a perfect example of, like, yeah, there's probably something off the shelf that could do something like this, but you've gotten not only, like, the content you want, but the user experience you want. You control it end to end, and you can build this, like, hyper-personalized software experience, which I just- it was not possible, um, or it wasn't, um, efficient to do, I would say, uh, until very recently.

   6. TTTomasz Tunguz13:46

      Yeah, it fits, it fits the workflow, my workflow like a glove, right? And anytime something comes up and changes, like maybe there's a section that's out of order, like we found, I can just go into Claude Code and update it, and it'll be done in 15 to 30 seconds, right? And, you know, I really wanted an email of this every day, and that was straightforward. So I agree with you. I think we're at a place where the marginal friction to achieving a glove-like fit with little utilities that maybe you wouldn't have paid for in the past is now, um... It's just so, uh, it's so quick, right? Like, you-

   7. CVClaire Vo14:18

      Mm-hmm

   8. TTTomasz Tunguz14:18

      ... you're just entering a couple of emails, and it'll be done.

   9. CVClaire Vo14:21

      Yep. You've seen the doom and gloom headlines: AI is coming for your job. But the reality is a little bit brighter. In Miro's latest survey, 76% of people say AI can boost their work. It's just that 54% still don't know when to use it. As a product leader and a solo founder, I live or die by how fast I can turn fuzzy ideas into crisp value propositions, roadmaps, and launch plans. That's why I love Miro's innovation workspace. It drops an AI copilot inside the canvas, so stickies, screenshots, and brainstorm bullets can become usable diagrams, product briefs, and even prototypes in minutes. Your team can dive in, riff, and iterate, and because the board feels like a digital playground, everyone has fun while you cut cycle time by a third. Miro lets humans and AI play to their strengths so that great ideas ship faster and happier. Help your teams get great done with Miro. Check out miro.com to find out how. That's M-I-R-O dot

7. 15:31 – 17:34

   A workflow for generating blog posts from podcast insights

   1. CVClaire Vo15:31

      com. Okay, so you have cr- taken all this content, um, including amazing content from the Lenny's Podcast Network, and you're processing it, you're extracting themes, you're extracting quotes, you're finding companies that may be interesting to reach out to. You're at least drafting Twitter posts. We will see if those actually get posted, um, you know, in production. And then let's talk about your second workflow, which is you extract insights that might be interesting for you to write about or add your perspective on, and then you actually turn those into drafts using AI.

   2. TTTomasz Tunguz16:08

      There's a lot of stuff that's happening in the ecosystem, and every once in a while, I like to write about what somebody said in a, in a podcast, right? Um, uh, and I think today, like, there-- I was looking... The GitHub CEO was actually interviewed, and so Matt Turck interviews, who's at, uh, another venture firm, interviews Thomas, and he talks about how AI and coding is the future. And so what I really wanna do here is, let's suppose I really wanna have a blog post that was tied to this. So what I can do is I say, like, "Okay, I have this podcast generator," and I'll show it to you in a second. And w- what I'll do is I'll take as context the transcription of that podcast, which is here, um, and then I'll define an output file, and then I'll give it a little prompt, which is like, you know, he said this quote, which is actually within the podcast summary. Everything that I can easily replace with a single prompt is not going to have any value. It will have the value of the prompt and the inference and the tokens, but that's often a few dollars. And I'll tell it, "Okay, go look for podcasts that are related to this," and I c- I've categorized them, uh, as AI. And then here, actually, there's a bug, so [chuckles] demo fail. I was trying to fix it before I got on the video, but the searching for the relevant blog post is failing, and I need to figure that out. It's, it's run through, um, LanceDB, a vector embedding database, locally. And then it'll generate a blog post, and I'll show you the prompt in a second.

8. 17:34 – 18:25

   Using the “AP English teacher” grading system for blog posts

   1. TTTomasz Tunguz17:34

      And the best... Well, one of the techniques that I found the most effective when generating blog posts is to ask it to grade it like an AP English teacher. And this goes back to my history. I remember not really loving to write until I took a class with, um, an Army veteran, and, uh, he taught me to really love to write, and he was my AP English teacher. And so I really like receiving feedback in that way. I grade it on a letter grade, and then tell me what I could improve, and then w- I'll iterate with the model until I get to an A minus.

   2. CVClaire Vo18:06

      Got it. And so just before we go into the actual writing, and I'd love to see a little bit of this AP English prompt, are these two pieces connected? Your podcast summaries, do those go into this VectorDB that can then be searched through for relevant other podcasts if you're writing on a topic? Like, how does this all come together?

9. 18:25 – 22:00

   Challenges with matching personal writing style using AI

   1. TTTomasz Tunguz18:25

      Yeah, so right now, it's just the blog posts that I've written in the past, the 2,000 blog posts or so, that go in. And the major reason I add-... those as context, is I'm trying to capture my style. And I have to tell you, like, that's really hard. Like, I have fine-tuned OpenAI, I have fine-tuned Gemma models [chuckles] and getting the voice, and you'll see it in the output, it sounds like a computer when it writes, even with that additional context, and, uh, it doesn't- the other thing that I, I have not been able to figure out is, I think it's really important in one blog post to link to other blog posts that I've written, just because the knowledge builds on itself, and obviously outside as well. But I haven't been able to figure out how to get it to link effectively.

   2. CVClaire Vo19:08

      Well, I, I think this is a, a common feeling with AI-generated writing. No one is satisfied with style, even when style is exceptional. I think I've seen examples, especially with some of the newer commercial models, actually writing really lovely prose and really lovely language. I- it's just, it's so personal what your style is and how you would write something, the rhythm in which you would write it, how would you punctuate and break line. All that kind of stuff is so personal, that I have, like you, had a very, very hard time [chuckles] getting it to write like me, and I think even harder, which is why I appreciate that you're not yet posting this. It cannot, it can't tweet like me. I can't, I cannot- [laughing]

   3. TTTomasz Tunguz19:51

      [chuckles] No.

   4. CVClaire Vo19:52

      The, the short ones, the short ones are the hardest, you know? Um, I guess they say that about, about writing, writing generally. Um, ha- have you felt like any of the models have done better or worse at writing like you, or is it just like they only get 70, 80% there, and I just accept the fact that I'm gonna have to rewrite things?

   5. TTTomasz Tunguz20:11

      The- well, they have different voices. Um, I don't think any of them are close. Uh, like, I think Gemini is more clinical, is the way that I'd put it.

   6. CVClaire Vo20:22

      I agree.

   7. TTTomasz Tunguz20:22

      Uh, Claude is more warm and verbose. Yeah, he's very, very garrulous, like, just wants to keep talking and wants really long sentences and really long paragraphs. Um, and, uh, OpenAI, I think the models each have a slightly different personality, so there I, I don't think there's, like, a single characterization. So I, I've been... I think I've been iterating to-- I used to use Claude 3.5 a ton, and I uploaded all of my blog posts in a project, and I, and then I'd have it iterate there. Now I can kind of do it with Claude Code or using this prompt, so that's a little less useful. But what I've found is, um, you really need to add your own voice, and then you need to tell the AI to keep the things that are wrong, right? Like, this, it's kind of a funny thing to say, but as you were saying, Claire, before, the way that you punctuate, I really like ampersands, right? And I like adding spaces before colons, and I like starting certain sentences with- or having little incomplete clauses, um, because I think they keep the reader moving. But an, an AI won't do that. An AI will only deliver you a grammatically perfect specimen. [chuckles]

   8. CVClaire Vo21:37

      Yeah.

   9. TTTomasz Tunguz21:37

      So-

   10. CVClaire Vo21:37

       We're, we're gonna have one, one very nerdy, uh, English language moment, which is I like to start paragraphs with a conjunction. I love a 'and' or a 'but'.

   11. TTTomasz Tunguz21:46

       Yeah! It pulls you in.

   12. CVClaire Vo21:48

       [laughing] So okay, you and I are gonna work, we'll, we'll build like a, an, a, a micro sass on [chuckles] good, good writing models, um, and prompts that, that people can

10. 22:00 – 26:13

    Tomasz’s three-iteration process for improving blog posts

    1. CVClaire Vo22:00

       use. So okay, so we accept that it's not gonna write exactly like you, but you've created this grading process to say, "Well, is it at least good?" And so I'm curious, can you walk us through how it gets to an A-minus 91?

    2. TTTomasz Tunguz22:16

       Yeah.

    3. CVClaire Vo22:17

       As, as an A-plus student, I don't know, a 91 would really stress me. [laughing]

    4. TTTomasz Tunguz22:21

       [chuckles] Well-

    5. CVClaire Vo22:23

       Uh, tell me how you kind of wrote the prompt, and then why you picked, like, A-minus as, as your bar.

    6. TTTomasz Tunguz22:29

       Yeah, for sure. Okay, so, uh, the way I wrote the prompt, I told it what I wanted. Um, and I asked an AI to critique... I think I asked Gemini to cri- to critique Claude's output. So I was kind of using a student-teacher critique model. And then what it does is, and we'll walk through the prompt in a second, but it goes through three grading attempts. So it reads a file, gives it a grade and a score, and then it-- the things that are the most important that I found, particularly for readers, are the hook, which is the first few sentences or the lead, you might call it, and then the last is the conclusion, and making sure it ties back, because then you have a complete, um, you have a complete post. And so it goes through this three times, right? And so you can actually see, like here, it gave itself a 90 and then a 91. Um, and then at that point, it basically was good enough. It was satisfied with the hook. So, um, if we, uh, let's see. If we read the blog post generator, um, you can see what it does at a high level, right? So it finds the blog post, it generates an initial blog post, grades it like an AP English teacher, improves, um, and then auto-generates a URL-friendly slug, so it actually writes it in the right format. And then it can use OpenAI or, uh, Ollama. And then the prompt is here. Uh, "You are an expert blog writer specializing in technology and business content." And then here I add in the blog posts, and it kind of shows the patterns. What it also does is, um, it dynamically calculates the number of paragraphs from relevant posts and uses Ollama to summarize the stylistic patterns of those related posts. So I might write a little bit differently when I'm targeting a Web3 or a crypto audience than, say, I might when I'm analyzing the public disclosures of a company, uh, Snowflake just announced their earnings, let's say. And so it's dynamically injecting that here. It shows a bunch of different examples, and then, you know, here's what I think makes my blog post tick, right? 500 words or less. I have, like, 49 seconds with a reader. No section headers. I ran a, uh, an analysis of dwell time-... as a function of how many headers there were, and it turns out headers were terrible for dwell time. People just bailed. Uh, flowing paragraphs, each paragraph transitions smoothly to the next. Actually, the AI consistently critiques my transitions-

    7. CVClaire Vo24:49

       Mm

    8. TTTomasz Tunguz24:49

       ... and says they're too harsh. And going back to the A-minus point that you made before, I think I lose five or six points because of my transitions, because they're abrupt. And then, you know, limit each paragraph to, at most, two long sentences. Uh, and then the structure of the blog post.

    9. CVClaire Vo25:05

       I, I think this is a really interesting t- story at the top, and I wanna make sure people don't miss it. I've seen this before, which is like, "Take this example and describe it back to me and use it." And so you're saying, "I'm writing on this topic. Go find the blog posts like this topic, analyze them for format, like what is... What is the structure? How am I writing things? And match, stylistically match this subset of, of my blog posts," because I do vary style by topic.

    10. TTTomasz Tunguz25:34

        Exactly right. Exactly right.

    11. CVClaire Vo25:36

        Good thing. Okay, and then, t- two sen- I was not expecting this, two sentences per paragraph thing. I, [chuckles] I like it.

    12. TTTomasz Tunguz25:43

        Yeah.

    13. CVClaire Vo25:43

        I have one more question for you as somebody who did take AP English, so this is, um, perfect for you. Did you actually... Do they publish the AP English, like, grading standards for the tests? Like, did you integrate any of that? Is it just sufficient enough to say, "AP English teacher?" I'm just curious how deep you went.

    14. TTTomasz Tunguz26:01

        Yeah, I just said, "AP English teacher." I figured there are enough people leaking, [chuckles] - leaking about. Either like the scoring rubrics or essays that scored fives or whatever it was-

    15. CVClaire Vo26:11

        Got it

    16. TTTomasz Tunguz26:12

        ... that there's good underlying

11. 26:13 – 28:16

    The grading prompt and evaluation criteria

    1. TTTomasz Tunguz26:13

       data.

    2. CVClaire Vo26:14

       Okay, so this is for writing it, and then what about for grading it? Do you have that prompt?

    3. TTTomasz Tunguz26:20

       Here's the grading prompt. So you're an experienced English teacher, here's the letter grade, numerical score, and then here are the evaluations: the hook, which, you know, argument clarity, evidence and examples, paragraph structure, conclusion strength, overall engagement.

    4. CVClaire Vo26:37

       Got it. And have you ever gotten Bs and Cs on grading?

    5. TTTomasz Tunguz26:41

       [chuckles] Yeah, for sure. [chuckles]

    6. CVClaire Vo26:43

       ... You seem like getting, like, 91%. I, I always wonder about this because I do think these models are positively inclined towards telling you you've done good work. I've found that consistently. I've always had to say, "Be more harsh, be more critical. Call out where I'm doing things wrong." So I'm curious, do you actually get high variability in these, in these gradings, or, you know, what has been your experience?

    7. TTTomasz Tunguz27:05

       Yeah, absolutely. So another... So this is one pathway for... I mean, the podcast-to-blog post data pipeline is one pathway for generating blog posts. Another one is just an idea comes to me.

    8. CVClaire Vo27:15

       Mm-hmm.

    9. TTTomasz Tunguz27:15

       And so then what I'll do is I'll just literally dictate. Um, I'll dictate, I'll put it in, and I'll pass it into the blog post generator and then have it grade, and there I've seen C-minuses, right?

    10. CVClaire Vo27:25

        Got it.

    11. TTTomasz Tunguz27:26

        Um, yeah.

    12. CVClaire Vo27:27

        So it's easier when it's grading itself and [chuckles] a little harder when it's grading you. This is super interesting. And then, in the... You, you do it three loops. Do you also get high variability between the loops? Do you find that that, that three-time process is actually additive to the evaluation?

    13. TTTomasz Tunguz27:44

        I do. I think I often see the first one, like a mid- what, like a 91, and then the second one will dip into the B, B-plus range, and then it'll pop back up.

    14. CVClaire Vo27:54

        Yep.

    15. TTTomasz Tunguz27:54

        So it's a little bit explore, exploit.

    16. CVClaire Vo27:57

        Yep.

    17. TTTomasz Tunguz27:57

        And again, most of the time for me, it's around those transitions, and most of the time, the verbosity of those transitions that the AI injects is just, like, catastrophic. I mean-

    18. CVClaire Vo28:06

        Yeah

    19. TTTomasz Tunguz28:06

        ... it doubles the length of the, uh, blog post. Um, and then the third, the third iteration tends to then kind of rein- reinforce the brevity.

12. 28:16 – 30:28

    AI’s role in writing education

    1. CVClaire Vo28:16

       Got it. And, um, my kids are too small for AP English to be something that I have to worry about yet, but meta question, you know, everybody's so worried about students using AI to write. And this seems like such a more fair way, [chuckles] way to evaluate writing. I'm curious, do you think we're gonna see more and more of this si- this type of evaluation in academic setting, and do you think teachers could benefit from, you know, checking their own work when they're grading these things that are a little harder to put quantitative or qualitative feedback against?

    2. TTTomasz Tunguz28:49

       Yeah, I think it's a great first-pass filter. Like, 80% of the work, what's going on grammatically, are you using sentences and conjunctions, and do you have dangling modifiers and all that stuff? Like, I think that, um, the rote analysis of the logic of that language, uh, should be handled by an AI, right? And then I think there's this other part, which is the stylist... I mean, you look at... I was reading e.e. cummings' poems last week, and you look at the creativity of some of those poems. Um, and I, you know, I think it only comes after you have the mastery of the language, but you'd want, you'd want teachers to be free to, uh, champion that or encourage it. I think it's really just, just, just as important.

    3. CVClaire Vo29:34

       Yeah. So for the students listening, you know, I still think it's good to learn to write, uh, to read a lot, to learn to write, to write yourself. And if you're looking for a place to practically apply AI to your writing work, maybe it's as a, a first-pass grade. Say, "If you were my teacher, how would you grade this, and what feedback would you give me?" As opposed to, "If you were me, how would you write this?" [chuckles] Maybe that's the right way to get students starting to use AI in a practical way that still allows you to develop these hard skills that I think are gonna be, continue to be super relevant.

    4. TTTomasz Tunguz30:07

       Could not agree with you more. I mean, oftentimes, I don't know about you, but I'll run into writer's block, or I'll have an idea that I really want to convey, but it's just a soup in my mind, and there an AI will help you iterate and refine, and often it'll give you the, the germ of an idea, and then you'll take it and add your specific lens to it. But, um, but yeah, I think it's a wonderful learning tool because you have the feedback so quickly.

13. 30:28 – 35:14

    Final thoughts

    1. CVClaire Vo30:28

       Yep, exactly. Okay, so you have shown us, just taking zoom back, 30-something podcasts you process on a daily basis. You create summaries, you extract themes, you extract tweets, you extract topics. Those topics then go into another, um, Python script that writes a blog post based on some other relevant blog to posts in your own, um, uh-... blog, writes the blog post, on-demand AP English teacher to grade you three times, and then you take the final pen, and then as AI posts, like, do you have it just like an agent going, ba-doo, boop, send hit or you- [chuckles]

    2. TTTomasz Tunguz31:07

       No, that I don't. [chuckles] That would be awesome. But no, that's, that's still done the artisanal way, point and click.

    3. CVClaire Vo31:15

       Okay. You are still copying and pasting with your human fingers.

    4. TTTomasz Tunguz31:19

       Mm-hmm. Yeah.

    5. CVClaire Vo31:19

       Okay, this is a great, super practical process. Um, I'm even thinking about ways I can do this to identify future podcast go- guests or, um, topics that people might wanna see, so you've given me some inspiration. I'm gonna ask you two wrap-up questions and then get you out of here, back into your terminal. First question, I was reading your 2025 predictions, and you said this is gonna be the year we see a 30-person, $100 million company. And I'm curious, when you... in your mind's eye, when you imagine that company, what does it l- who's in it? Like, what are they doing? How are they operating? What do you imagine that company looks like?

    6. TTTomasz Tunguz31:55

       Yeah, I think it's probably there's a CEO who's a product person, there's an engineering team of 12 to 15, and then there's probably a couple of customer support/dev rel people, and maybe there's a salesperson, maybe who's closing some of those bigger contracts, and then a s- solutions architect as a function of the kind of company. But it will be predominantly software engineering, and then I think the go-to-market motion is PLG, Bonanza, just massive adoption.

    7. CVClaire Vo32:24

       And do you think those software engineers are largely still focused on product building, or do you imagine that those software engineers are also enabling the company with tooling and automations, and figuring out how one salesperson can do the work of 20? I'm just curious how you think that's gonna shake out.

    8. TTTomasz Tunguz32:40

       Oh, absolutely. I, I think that's right. I mean, uh, you... We were, we were kind of talking about this, but, like, the, the ability of a person to come up with a demo and then use AI to critique the demo and test, uh, is now so fast, and the ability to take that code and basically move it into production really quickly is also incredibly fast. So I do think there will be a pretty significant, like, internal platforms enablement function. And whether that's kind of 20% time for a bunch of engineers or a dedicated team of two or three people, a huge amount of leverage there.

    9. CVClaire Vo33:12

       Yeah, I, I completely agree. Okay, and then last question: When your AI is grading you unfairly or writing terribly or making very long transitions that do not, um, sound like you, what is your prompting technique to get AI to listen?

    10. TTTomasz Tunguz33:28

        [chuckles] I have two AIs duke it out. [laughing]

    11. CVClaire Vo33:32

        [chuckles]

    12. TTTomasz Tunguz33:32

        And so I have, like, a little example of, like, "This is the input, this is the output that you gave me, this is the output that I want," and then I have Gemini and Claude duke it out and finally kind of decide on... Um, and I'll use a little script to do that, where they'll finally polish a script. It doesn't work all of the time, but I do think switching models helps a ton. It, it creates a level of generalizability that, uh, I haven't been able to replicate as a human.

    13. CVClaire Vo33:56

        I, I agree, and I will give you a How I AI tip from a previous gr- uh, previous guest, Hillary, who, like, negs the models to each other. So they're like, "Gemini, look at this garbage." [laughing]

    14. TTTomasz Tunguz34:08

        [chuckles] No way.

    15. CVClaire Vo34:08

        "How to..." And then they're like, "Claude, look at this trash OpenAI gave me. Like, surely you can do better than this." [chuckles] That's where she calls it Mean Girls, and she's like, "I Mean Girls the models-

    16. TTTomasz Tunguz34:19

        Wow

    17. CVClaire Vo34:19

        ... and get them to compete with each other." So maybe you can create a, a, a Python-based terminal script to, to do that and then share it with, with our audience. Open sou- open source that thing.

    18. TTTomasz Tunguz34:29

        Uh, great, great idea for a weekend project this Saturday. [chuckles]

    19. CVClaire Vo34:32

        Well, this was so helpful. Uh, where can we find you? How can we be helpful to you?

    20. TTTomasz Tunguz34:37

        Oh, I'm on tomtunguz.com, and, uh, if you're starting a company within the AI ecosystem, I'd love to hear from you.

    21. CVClaire Vo34:43

        Great. Well, thank you so much for being here.

    22. TTTomasz Tunguz34:44

        Thanks for having me, Claire.

    23. CVClaire Vo34:45

        [upbeat music] Thanks so much for watching. If you enjoyed the show, please like and subscribe here on YouTube, or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review, which will help others find the show. You can see all our episodes and learn more about the show at howiaipod.com. See you next time. [upbeat music]

Episode duration: 35:14