GPT-6 Astra blew away every one of my benchmarks

1. 0:00 – 3:44

   GPT-6 Astra overview

   1. CVClaire Vo0:00

      We are going to the stars today because yes, today GPT-6 Astra is live. Well, it's live for Daybreak customers. It will be live for the rest of us very soon. I was lucky enough to get some early access to test the model, and this has been the model in the last six months, and, you know, maybe I sound like a broken record, but this is the one that has made me actually feel more ambitious. It has crushed some of my standing tasks that Fable couldn't figure out, even my beloved 5.6 Sol couldn't figure out. This model, GPT-6, one-shot a bunch of them. I'm gonna tell you what this model is, what OpenAI says it's great at, and I'm gonna show you some pretty incredible things I've built with it, including, yes, hacking this dumb computer again. Let's get to it. Okay, I don't wanna spend too much time on the blog post. We can link to it in the show notes. You all can go read it. TLDR, state-of-the-art on basically everything. We got the science, math, coding, knowledge work, not being annoying, being fast, computer use, all the things. So, OpenAI is positioning it as its most intelligent and aligned model. Huge, um, jumps, uh, particularly on frontier math, a big jump there in mathematical ability. The big thing that they're calling out is computer use and the use of software. So you're gonna see this in today's episode, but it is so good at computer use, um, not just browsers, but Excel, Unity, Power BI, Blender, Jupyter, document editors, like kind of everything. So, uh, we're back. SaaS is back. Product's back. Buttons are back because now Astra can use it. Um, it's really good at producing work artifacts. They've done a lot of work to make work artifacts match your style, match your brand, match your voice, et cetera. And then again, autonomy, steering, all that kind of stuff is, is very good. They've really focused a lot on kind of time saving on common tasks, again, because of computer use. And so this is a little, like, sneak peek into, I think, like we're all spending a lot of money on tokens, and these models are feeling pressure to come up with an ROI model. And, and I think you're gonna hear a lot about them talking about Astra in terms of computer use and time saving, right? Like apartment hunting can go down to 10 minutes from six hours. Um, I'll show you an example of generating thumbnails. Again, something that takes us like an hour, um, down to a couple minutes, completely hands-free. And so, um, a l- a lot faster, um, a lot more efficient, and kind of able to do what they say real work. They really call out presentations, spreadsheets, documents, and software as great use cases for Astra, but let's look a little bit at kind of some of the numbers. So Automation Bench, again, double Sol's reported scores. So, you know, a lot higher on, um, automating tasks. Let's look at coding. DeepSwee, a little bump up here against 5.6 Sol, but again, um, combined with computer use, you're gonna be able to do a lot more really interesting things. Um, math is a big story here, so big jump in frontier math tier four. Um, and then let's talk about price and can you get it? So, um, it's rolling out. First, it's gonna go to enterprise customers with Daybreak access. Um, I got a little bit early access, so I could test the model and tell you all about it. Um, but over the coming days, it's gonna be rolling out to GPT Plus, Pro, and Enterprise in the API and in AWS. And then pricing, $10 per million input tokens and 50 per output tokens, and it... Fast mode will be fast. Um, so it's a great model. You're gonna wanna use it. And I just wanna get that out of the way so you know what it is, and then let's get into how amazing

2. 3:44 – 9:08

   Browser/computer use test on my CRM

   1. CVClaire Vo3:44

      it is, starting with computer use. So now that I've told you a little bit about what Astra is great at, let's see it in action. Now, the number one thing I covered when talking about the high level of the model is how good it is at browser use, and I am telling you, I am a browser use maxer here. I'm a computer use maxer here. I love browser use. I love computer use. But... And I've been using it a lot in Codex generally, but I will say Astra has taken my computer use to the next level. I wanna show you a couple examples of some things that I've done with computer use that I think are really, really, really useful. The one I was really surprised by is how good it is at complex web UI and managing pretty complex nodes and interactions and doing it really well. So let me show you an example of what I mean here. So this is our cxo.dev CRM, and we are starting to get leads. We launched this business last week, and we're getting a bunch of inbound. And we want to filter those leads and then determine whether or not they should be sent to me or sent to my business partner, Zach. And once they are determined if it should be me or Zach, and it's, like, I get the big fancy enterprises, he gets the, like, technical startups, and we kind of divide and conquer responding to leads one-on-one. We haven't given that to the agents yet, but we're gonna do it live here. So we use this kind of built into our CRM, Adio, node-based workflow builder to build out the assignment logic, do things like prompt, um, to determine Claire or Zach, and then route it to different Slack messages and different CRM attributes. And I found myself sitting here and, like, futzing with these buttons one-off. I couldn't figure out, like, how to infer the right objects from the responses, the AI responses. I was just getting really frustrated. I spent an hour here. And then I was like, "Ah, computer use. Of course computer use can do this." And so I wanna show you what that looks like. Okay, so instead of dragging and dropping all this stuff, I'm gonna bring up my chat where I talk about the CRM with, um, Codex, and I would say, "Hey, I have a workflow open to route leads to Zach or I. I want As part of this flow, to generate with a custom agent AI an email that is custom to the lead submission authored from either Zach or I, depending on who it's routed to, and, um, invite them to, uh, schedule with our calendar link, plus a bit about why we're excited to chat with them. Can you update the workflow in Chrome to do this? Thanks. Okay, so what we should see Codex start to do is actually open up Chrome, take over Chrome, and start building out new nodes, new logic, new prompts, and route all of this stuff, reconnecting the right nodes, and here it goes. So you can see it's adding a new object to the AI generation that we're doing, um, who the email's going to, what the email subject is, um, what the email body is, I'm guessing. Look, there's no hands. I'm not doing any of this, and it's going ahead and going in this, like fairly complex node-based app and figuring out where to make the changes, include things. And I think ideally, it's going to show an email draft from Claire or from Zach to review and for us to ultimately send. And so that's gonna send to our Slack channel, and then we can copy and paste that and, um, email it to the, um, prospect. You know, the other thing, what I'll probably do here is have it actually send the email after we feel like this is really good. But again, this kind of thing, just like typing and thinking takes forever. And before Astra, I mean, really before GPT-56, but certainly before Astra, I really just didn't feel like browser use and computer use could navigate these nodes as well. And now I've found that it really can. And I just am hands-off my computer all the time now, and it's a really fascinating new way to think about interacting with your computer, interacting with websites. And I think about this. We, you know, we all said no UI is the next UI, and we had all these conversations about MCPs and CLIs. And what's so fascinating is now we're at this moment, we're actually like, UI's back, baby. Like, SaaS, this might be your savior because humans really do like a button, and now [chuckles] that you can get AI to click buttons for you and use a UI for you, you sort of don't have to think about these CLIs and MCPs necessarily if the right expression of your product is more of a UI-based thing. So I really do think computer use is kind of incredible. I'm gonna let this keep running. I'm gonna give another example of something kind of completely fresh that I haven't done yet,

3. 9:08 – 13:00

   Flora thumbnail generation

   1. CVClaire Vo9:08

      but I think it's gonna do really well, which is I'm gonna say we do a lot of thumbnail editing for our podcast in Flora. I have Flora open in Chrome right now with all of our podcast thumbnail upscaling workflows. I'm gonna drop you a couple images from the podcast I'm recording live right now, and can you generate the images that our designer Naomi would need to actually generate the thumbnails for this podcast? And then I'm going to take-- I took some photos, uh, earlier today, so I'm gonna drop, right before I started this, I'm dropping really high quality photos of your friend Claire looking super excited about Astra, 'cause I am. And I'm gonna send this, and we're gonna see if it opens up Flora, which again, is sort of this like node, node workflow-based AI editing tool for images and videos. I'm gonna see if it can navigate that because, again, I haven't done this before, and so I wanna see if it actually works and can do that well. So it's gonna inspect the thumbnail workflows that I already have open in Flora and then use these three photos to prepare the image assets Naomi needs. You can see here, this is, um, our kind of like workflow. It looks cuckoo bananas. This is not super well organized. As you can see, we take like kind of normal photos of me recording the podcast or a guest recording the podcast, and then we make them like glowy and look fabulous because I'm sorry, but that's what YouTube says you like. It says you like this like glowing purple hair Claire. Um, I wish you liked normal office lighting Claire, but you like purple glowy hair Claire. And so what it's gonna do is it's actually gonna take these new images that I dropped, navigate this app, import these images, and hopefully generate us the videos that we want that you'll see today, um, when you come to the podcast. So I'm gonna let that run, and we're gonna come back to it once the nodes are being generated. Okay, so let's watch what it's doing. It took this imported photo of me that I just took in Photo Booth. It's going through and drawing a node, selecting the image generation model that it's gonna use. We use gpt-image-2. Um, we used to use nano banana, but we actually really like gpt-image-2. It's selecting aspect ratio. It's putting in the prompt based on other prompts we have, and it's gonna go ahead and generate that, and I think it's gonna do it for all three of these images. And again, this is something that our designer, Naomi, who's fabulous, does by hand every time. It's sort of mindless, and it takes a little bit of effort to drag the nodes, get it all organized. Also, we love Flora, but it's kind of like chaos reigns here, so anything that can help us reorganize these nodes and keep them in check can be very, very helpful. And so what we're probably gonna do is have Codex now generate all these images, and then Naomi's job, like all humans, is just to have good taste and make sure that the model doesn't make my face look insane or give me extra fingers or, um, give me like an uncanny valley look. But you can see here- It started generating one image. It's gonna go ahead and generate the next. And so you can just think of this as two examples, this, like, complex CRM workflow node builder and, um, using Flora to generate assets and do prompting as something that really benefits from this UI, but does not benefit from humans using their fingers and typing prompts and dragging things to make the UI. And this is where Astra really, really shines. And so I do spend about 90% of my day sitting here watching Astra use my computer and specifically use Chrome to navigate websites and applications that I really love. As

4. 13:00 – 15:20

   Browser use for QA

   1. CVClaire Vo13:00

      a side note, another use case that I use for Astra and browser use that I think is really helpful for software developers in general, and we'll get to coding in a minute, is just QA. And so recently, I was working on stabilizing the main ChatPRD, um, chat experience with A- uh, with Astra, and we were chasing down some, like, race conditions and some weird edge case things in streaming persistence and all this kind of stuff. And we finally got the code shipped and the PR shipped, and I was like, "Can you just, like, test this in Chrome?" And what was really awesome is it just pulled up the preview branch, and then it started clicking through testing, sending chats, and what was really helpful about browser use here is it was inspecting the console, checking for error logs. It was doing things like race conditions. It would, like, refresh Chrome. It would do all this stuff that would be very tedious and hard to execute as a human, but was super useful as a QA. You can see here it actually ran for an hour and 45 minutes, basically, QA-ing this branch. It found a couple issues when you were navigating. It fixed some things. It specifically looked at some, um, race conditions and was really awesome. So if you are not using browser use for QA, please do. Um, I'm just gonna go wrap our computer use section [chuckles] by looking... It found and made the links. Kind of incredible. I'm actually just going to Slack Naomi right now and say, "Good news. Astra made the thumb photos itself in Flora. Super cute." And then I'm going to send her an example of it, and, like, this is so funny because think of a perfect example. She's got an appointment, um, we need this thing done because you all like your model reviews ASAP, and Astra made a great image for our thumbnail. And so, um, this is just a really great example of a model and browser use and computer use just taking toil off humans, even in the creative fields, and letting you do your work a little bit faster and letting you impress your customers. You all are my customers. You're my clients. You're my audience. I love you, um, a little bit more. So very, very cool example. I

5. 15:20 – 18:36

   Coding: ChatPRD product intelligence feature, finally one-shotted

   1. CVClaire Vo15:20

      love Astra. I love browser use. But let's get to some other fun stuff. Okay, let's talk about coding. Yes, obviously, it's great at coding. Um, but specifically, it has broken through a couple software engineering challenges that I have not been able to crack thus far with GPT-5.6 or with Fable or with Opus. Specifically, I, for ChatPRD, have been working on this feature, this product intelligence feature that, like, sort of, like, sucks up all your product intelligence from, like, Intercom and Granola and Linear and GitHub and kind of tells you what's going on, what's trending, what you need to do. And in addition to creating those insights and trends, it generates this sort of like auto wiki of the state of your product. And I have thrown every model at this. I'm pretty smart. I kind of like architecturally know how it needs to work, but it is complex. It has to ingest data, has to decide what it durably stores. It has to process it. It has to de-duplicate insights. It has to generate a wiki. It has to verify that wiki against external sources, and, like, I-- Fable did insane things with the architecture. 5.6 just, like, got stuck on quality of insights, and I am telling you all, Astra one-shotted. And I was like, ki- it was like, this has been my experience this entire model, which has been like I've-- It's been so hard to do these, like, very specific tasks, and I've tried them over and over and over again for six months, and then all of a sudden, Astra started to one-shot it. So I have been working on this product graph, product intelligence forever, and finally just gave it to Astra. I'm like, "Can you... I have a branch open. Can you fix it?" And it built it. So it insi- it, you know, ingests, um, data from a bunch of different sources, Intercom, Granola, Linear, et cetera. It pulls in those insights. It tells me what my top three priorities as a product person should be. Um, AI stalling, that was that, um, race condition bug that we were figuring out. Self-service cancellation is broken, and then people want on-prem Jira, so Jira friends, please. Sponsors of the pod, people want on-prem Jira integration. It generates these kind of graphs over time. It, it tells you what opportunities, and I am telling you, I could not get any other model to get to this quality of insight out of a set of data. But my favorite part is it actually generated the auto wiki well. So what this does is it sucks up all the information it can find about your product, in this case, about ChatPRD, and it went and looked at GitHub, it looked at our documents, and it just basically explains current function of all the product, recent changes, customer context, things that we need to fix, and then it exposes this via MCP. And I am telling you, this was like, I would call it three shot. One shot got it to 90%, and it was, like, two or three other prompts, and finally it got it working. I had to polish the UI a little bit, but this is one of those things that just, uh, y- none, none of these models could do really well out of the, out of the gate. It wasn't that they-- I couldn't technically explain how I want it executed. The architecture and whatever decisions it made on how to do it just resulted in poor quality, and it really... Astra really figured this one out. So it has super accelerated my personal product development, software development on ChatPRD.

6. 18:36 – 22:23

   Hardware hack: Divoom MiniToo CLI and live streaming display

   1. CVClaire Vo18:36

      But more important, it's let me build some fun stuff. So let's talk about coding, but make it fun. So you all know that I have been victimized by this thing. It's a, um, Divoom MiniToo. It is really just like a Bluetooth speaker. It comes pre-installed with proprietary software. You can't-- There's no API. You can't crack it. You can't hack it. And GPT-55 was the first one that even let me do the most basic thing on this little computer. This has been... This is my, my Everest. And already, I mean, the quality of what I've been able to flash onto this little device is amazing. Look, it's like Claire's little lab. It's figured out, um, the pixel algorithm, which I had to, to reverse engineer. It was not easy. Um, so it's, it's already doing cute art. But let me show you what Astra was able to do here. So Astra built a little app that allows me to connect live to, um, this device. I can draw on the device using my mouse. I can change the thing. I can type, "I love my subscribers." Uh, it's not gonna spell check for me. I can do all that live, live streaming. But even cooler, I had Astra build me a CLI, um, now that it's totally figured out this thing, um, so that it can talk to me live about anything it wants. And so I can say, um, "Use the CLI again to tell our audience about the most recent episode of How I AI." And so it can now go look. Now it's saying, "Codex is working." Very cute. It's going to think really hard about what it's doing. It's gonna find the latest How I AI published episode, and it's going to stream an audience-friendly introduction to the CLI. I mean, again, it took me forever to get this thing to say hi with these previous models, and now I am truly able to, um, do anything I want on this. So it's gonna say, "Codex is working." It's reading the Python that is the, um, script, and then hopefully, fingers crossed, it will stream a message to you all beautiful subscribers about our most recent episode of How I AI, and let's just see if it gets it right. It got it right. The most recent episode is GrokBot versus OpenClaw, but let's see if it can actually stream it to little computer. Um, I'll show it to you all, and we'll see it come in. Here it goes. It's streaming. So it's saying, "Claire shows us GrokBot." It's streaming in the episode information, and then it should show us a super, super cute little image, um, about my many GrokBots. But look at that. It just works totally out of the box. Look how cute that is. It even animates. Okay. This, again, is my Everest. It also says, "Inside PR support, SOC 2 monitoring, and moving agents from OpenClaw. 36 minutes, a real agent tour. Find it in your podcast app today." This thing, we did it, folks. Um, next I gotta root it, so that's next. The other really cool thing that I did with this that you all haven't seen is I hooked it into my Codex hooks, so now it can tell me when Codex needs something, and that brings me to

7. 22:23 – 24:24

   Building an AIM-style Mac app

   1. CVClaire Vo22:23

      my next use case. Very fun. Again, just coding things that I could not do before that I can do now. Let's look at it. It is building me a Mac app that turns my Codex threads into a '90s-style AIM messenger. For those of you who are young and beautiful and were maybe not even born when AIM was around, um, AOL Instant Messenger, it's how we olds used to chit-chat with each other on the internet, and it looked like this. It looked like this. And look at this thing that Astra made for me. Okay, so it made a '90s AIM Instant Messenger-style wrapper around all my live Codex threads. So all my buddies are different threads. Here's the Divoom little mini app, um, so I can open up the chat that we're chatting there. It said it streamed the September 2 episode on GrokBot versus OpenClaw. I can go back and send it another message. I can look at different things, like I built this Astra podcast re-review. So I can chat with Codex as if it's my little AIM buddy. It's so stinking cute. It lets me have my screen name. Here's my away message, "BRB, letting my agents cook." And then you can see here, I can say to that thread I was just working on, "Great job. That worked awesome. I love it so much." And it can... Oh, it's already got an active th- active writer, so it knows when it's actually working, which it is right now. It won't let me write. Little error. Again, this was one shot. But again, this is the kind of thing that I want your mind to go to, like things that you never thought were possible interacting on your computer, building software, building custom things, you can do one shot right now. It is so, so, so fun, so awesome. This brings me to,

8. 24:24 – 28:52

   Blender and 3D assets: Barbie Bench and the kids’ family app

   1. CVClaire Vo24:24

      um, another thing that you'll probably see a lot of the other reviewers do because a few of us that had kind of early access to the model, um, this was like spreading like wildfire, is it's really good at generating 3D assets in Blender. This is something that they call out in- Their blog post, but is something that it does quite well. And so this is gonna be a step change in making 3D games. And I have another bar- benchmark. In addition to have Tiny Computer Bench, I have Barbie Fashion Designer Bench. And you all, again, are young, beautiful people, so you don't know this, but there's this game. It was called Barbie Fashion Designer. It came on a CD-ROM. And you [chuckles] had this 3D Barbie, and you could design her clothes and, like, pick colors and prints, and then you could actually print out clothes on this, like, printable fabric, and then you could, like, tape it together and put it on your Barbie, and then she walked, like, a 3D runway like this. And I know all of you are out there making space games, and making race games, and making RPGs, and making all sorts of stuff. That is not for me. I am a girl. I want 3D Barbie Bench. I want Barbie Bench, baby. You guys, I've tried over and over and over to build this, even before Fable, even before GPT-5. This is what Fable did real uggo, really just terrible, terrible stuff. And so this was my bench. I was like, "Can we use Blender and get Barbie Bench going?" And while I will say it is imperfect, it is certainly better than it was. So here's our Barbie 3D, um, gal that we made here in Blender. You can see a step up from this. So [chuckles] we've made some progress. But let me show you exactly the game it generated, which I think is super cool. Okay, let me show you what Barbie Bench did. So now we have this stunning, gorgeous gal here, and you can pick different tops she wears. You can do shirts. You can change bottoms. Now, pants bench still not, uh, quite done. She's got these nice, like, Christina Aguilera, um, sort of chaps situation where her hips are showing. It's very chic. Um, shoes, you know, leave something to be desired, but, you know, who doesn't like an uggo shoe? And then accessories, again, look at this queen. Look at her glasses. Look at earrings on and off. A little bow looking cute. A little lock, um, as, as a purse. And then you can change her hair. Let's just... Ugh. Stunning. Stunning, stunning, stunning highlights slash pigtails slash whatever. Um, you can change her skin tone. She can be beautiful. Look at this. And then you can have her walk the runway beautifully. I mean, nothing better than this. Now, is this perfect? No. You're gonna see a lot of games that are a lot better than this. I think this, like, human 3D rendering is a lot harder. This was one shot, though, so I cannot complain. I wanna show you a different one, though, that I think is a lot more beautiful and a lot more interesting. So you all know that I have been working on this family app to bribe my children to do their homework, and, um, I asked Astra to turn this app into a 3D journey where my kids could walk to the library, for example, and say, "I read 20 minutes. Collect my reward." Or go to Moon Water Pond and see, um, what my weather is, my emotions are today. We're doing a lot of, like, SEL stuff in my house. So again, like, walking through this, and look how beautiful all this is. How so super cute. Um, you can use keyboard control, which I'm using right now. I'm sure, um, Astra would be really good at controlling this as well. I mean, truly one shot. I did not code any of this. I just actually said, "Can you make this a 3D, you know, journey where you can walk through different places?" And it did this out of the box, and it's so stinking cute. It's so cute. I would never be able to do something like this. Never, never, never. And Astra made it totally possible. I love it so much. It makes me so excited about what the future of building products is, what the future of building things for my ki- kids are, personal software, all those things. I am just psyched about Astra.

9. 28:52 – 32:19

   Summary: what Astra is great at and what to try first

   1. CVClaire Vo28:52

      Okay, and to wrap it up, during one of the breaks, um, our designer said, "I have to go to a meeting. I'm not sure if I'm gonna be able to do thumbs in time." So I did ask Astra to take over my computer and actually use Figma to make thumbnails for this week's episode. Let's see what it did. Look at that. YouTube classic. This face. Maybe this one will go live. Again, this has been something that computer use has been quite terrible at to date. I have not gotten it to use Figma very well, but look at that. It did it. Pretty incredible. All right, so to take a step back, Astra GPT-6 is a banger. What is it good at? One, it's good at computer and browser use. What should you use that for? Navigating complex UI, lots of buttons, lots of nodes, lots of little fields, things that you don't wanna fill out. Really, really exceptional at that. Um, I used it in my CRM to build new pipeline workflows. I used it in Flora to generate thumbnail images. And then you saw I used it in Figma to actually put those thumbnail images together so that we had thumbnails for this podcast. We'll probably polish them, but truly not bad. On the coding side, it's gonna help you get a lot more ambitious. So it one-shot a couple things that, in the past, these models just could not do. One was parsing through a lot of unstructured data, deduplicating and deriving insights out of that, and then generating a great UI for us in ChatPRD to share those insights with our customers about their own product. The second one, um, was this guy, um, hacking hardware. Please hack your hardware. Hack this. Hack your Bluetooth light bulbs. Hack, hack, hack, this guy. Um, just being able to interact with hardware and the real world is such a fun thing, and it was, it's been a step change in this model in my experience, and I've been working on this for a long time. Upleveling your ambition of what kinds of things you can code. So, like, website vibe coding's over. We can all do that. Let's do 3D games. Let's do Barbie Bench. I'm gonna do this 3D game for my kid. Or let's do desktop apps, like this AIM-style messenger that wraps Codex and does something pretty incredible and lets me interact with my Codex threads in this really fun UI. Again, something I would've never been able to build, and GPT-6 one-shot it. I really do love this model. I will just say, like, meta things, it's not annoying. Uh, it's kinda slow, but, like, not too slow. I'm happy with it. It gets the stuff done. This is gonna be my daily driver. I love her. So that has been my review of GPT-56. We will do a follow-up episode comparing 56 to Fable 5.1 and a couple other of the new models. We'll do the How I AI bench. Um, we'll do the cl- the blind Claire taste test. But for now, as soon as you get access to GPT-6, you should be using it, and please show me what you build. Thanks for joining How I AI, and I cannot wait to see what you do with this new model. [upbeat music] Thanks so much for watching. If you enjoyed the show, please like and subscribe here on YouTube, or even better, leave us a comment with your thoughts. You can also find this podcast on Apple Podcasts, Spotify, or your favorite podcast app. Please consider leaving us a rating and review, which will help others find the show. You can see all our episodes and learn more about the show at howiai pod.com. See you next time. [upbeat music]

Episode duration: 32:19