How to scrappily hire for, measure, and unlock growth | Crystal Widjaja, Gojek and Kumu

1. 0:00 – 9:40

   Crystal's path to product

   1. CWCrystal Widjaja0:00

      ... I felt like it was a problem that was very solvable. And we ended up renting a stadium to just hire, like, 60,000 drivers (laughs) in a couple of weeks. So, I think, looking back, it was certainly a risk. Like, when I got there, it was in a house, and I realized, "I've probably made a huge mistake." But we were growing very quickly already, even at that small scale of, like, 4,000 orders per day.

   2. LRLenny Rachitsky0:27

      (Instrumental music.) Crystal Wijaya has been leading product and growth teams at some of the largest consumer businesses in Southeast Asia, including Kumu, where she's currently the chief product officer, and Go-Jek, where she built and led the growth team through their early years of what is now the largest super app in Southeast Asia. To put this in context, Go-Jek completes more rides per day than Lyft, and more food deliveries than GrubHub, Uber Eats, and DoorDash combined. And it's the number one mobile wallet in Indonesia and Southeast Asia. In my opinion, American startups have a lot to learn from startups in Asia. And Crystal has been at the ground floor of some of the biggest successes there. In our conversation, we cover the biggest growth unlocks that Crystal has seen across the companies she's worked at, what growth investments usually pay off, and which often don't. We dig into growth models, a bunch of pro tips for accelerating growth, why most analytics efforts fail and how to avoid that, how to hire and structure your growth team, and we also talk about the nonprofit that Crystal started that aims to help young women get into STEM, called Generation Girl. Crystal is such a star, and I hope that you enjoy this episode as much as I did. And with that, I bring you Crystal Wijaya. If you're setting up your analytic stack but you're not using Amplitude, what are you doing? Amplitude is the number one most popular analytic solution in the world, used by both big companies like Shopify, Instacart, and Atlassian, and also most tech startups. Amplitude has everything you need, including a powerful and fully self-service analytics product, an experimentation platform, and even an integrated customer data platform to help you understand your users like never before. Give your teams self-service product data to understand your users, drive conversions, and increase engagement, growth, and revenue. Ditch your vanity metrics, trust your data, work smarter, and grow your business. Try Amplitude for free. Just visit amplitude.com to get started. Hey, Ashley, head of marketing at Flatfile. How many B2B SaaS companies would you estimate need to import CSV files from their customers?

   3. NANarrator2:38

      At least 40%.

   4. LRLenny Rachitsky2:40

      And how many of them screw that up, and what happens when they do?

   5. NANarrator2:43

      Well, based on our data, about a third of people will consider switching to another company after just one bad experience during onboarding. So if your CSV importer doesn't work right, which is super common considering customer files are chockful of unexpected data and formatting, they'll leave.

   6. LRLenny Rachitsky3:02

      I are zero percent surprised to hear that. I've consistently seen that improving onboarding is one of the highest leverage opportunities for both sign-up conversion and increasing long-term retention. Getting people to your aha moment more quickly and reliably is so incredibly important.

   7. NANarrator3:17

      Totally. It's incredible to see how our customers, like Square, Spotify, and Zuora, are able to grow their businesses on top of Flatfile. It's because flawless data onboarding acts like a catalyst to get them and their customers where they need to go faster.

   8. LRLenny Rachitsky3:34

      If you'd like to learn more or get started, check out Flatfile at flatfile.com/lenny. Crystal, thank you so much for being here. I've read a bunch of your stuff online. We've exchanged a bunch of emails and tweets. But this is the first time that we're actually chatting for real.

   9. CWCrystal Widjaja3:53

      Yeah.

   10. LRLenny Rachitsky3:53

       And so I'm really excited to... Yeah, I'm excited to learn from you and for folks to learn about you.

   11. CWCrystal Widjaja3:56

       We may have, like, crossed paths on Clubhouse, in the audio forums, or on the Twitterverse so it's really cool to see.

   12. LRLenny Rachitsky4:04

       Wow, I just remembered that. That is so right. I think we were talking about Reforge and, and Epbo. Is that right?

   13. CWCrystal Widjaja4:09

       Yes, that's right. Good times.

   14. LRLenny Rachitsky4:12

       Oh my God. Clubhouse days.

   15. CWCrystal Widjaja4:13

       The days when Clubhouse was a thing.

   16. LRLenny Rachitsky4:15

       Yeah.

   17. CWCrystal Widjaja4:15

       They have a thing too to learn about exponential decay.

   18. LRLenny Rachitsky4:18

       Oh, man. Okay, maybe we'll get to that. And we're gonna, we're gonna be chatting a lot about consumer growth and a bunch of stuff along those lines. But before we get into that, you have a fairly unusual path, and also geography, as compared to many of my other guests. And so just to set a little context, could you just kind of walk us through your, your kind of career path and journey from I think as an investment banker initially, and then currently as chief product officer at Kumu, and then living in Singapore also. So yeah, tell us all about your path.

   19. CWCrystal Widjaja4:45

       Yeah. I, I think my path was certainly an, uh, a nonstandard one. While I grew up in San Jose, the Bay Area, you could see companies like Lyft kind of emerging around the year that I was graduating college. But really it was, "How do I graduate college as quickly as possible? Because this is very boring." So I took a poli sci major. I am not a math or a computer science major. I didn't know what a consultant was, because I was just, like, trying to get out of college. I didn't realize people start looking for jobs before they graduate. So the last two weeks of school, I was looking on Craigslist, 'cause I was like, "Craigslist is how everyone gets a job, right?" I'm, like, a first-generation American student, so my parents could not help me at all with, like, "You should look into this company called McKinsey," or, "Here are all of the life paths that you have ahead of you." So I ended up taking an investment banking research job. And my job there was to figure out how to call startups and analyze their potential for VC financing or M&A advisory. And I barely knew what those words meant at the time. I ended up owning a huge Excel database of, like-130,000 rows, 60 plus columns. And because again, I am very impatient, I was like, "This is a terrible experience. How would I create a customer database?" And so I ended up kind of Google-fuing all of the work needed to build a MySQL database. I presented a plan and investment banking, surprise, surprise, is not very tech-forward. So they looked at my plans and they're like, "What is this MySQL thing? Isn't that super expensive? What is open source?" So I ended up leaving that job because I realized that if I wanted to get into something more tech, it would probably not be at a investment bank. So I di- took the investment banking strategies that I had learned there and kind of applied the same pattern matching to companies in Southeast Asia. So my family originally is from Indonesia. I thought, "I have a kind of safety net. I must speak Indonesian really well just by birth. So maybe that's a great country for me to look at." So I took kind of the approach of let's find a company that makes a lot of sense that I feel I resonate with, and I literally cold called emails at some companies. So Go-Jek being on that list, I literally emailed someone after googling HR at Go-Jek and said, "I'm willing to move to Indonesia. Take a bet on me." Um, and they actually did. So I got extremely lucky. Five years fly by insanely fast. I went through building out the data team from scratch. When you have all of the data, you know how much fraud you have in the system. So then I ended up building out the fraud and risk team, picked up performance marketing, and then it was like, "Okay, now we're ready to grow. So you have all of this data, now take on growth."

   20. LRLenny Rachitsky7:55

       Got it. You kind of, uh, were very modest about Go-Jek and the success of that company, and also Kumu where you work now. So just to kind of set a little context for folks that aren't familiar with these companies, can you share how big they are and how big of a deal they are in Southeast Asia?

   21. CWCrystal Widjaja8:11

       Yeah, they are, uh, pretty massive. So Go-Jek is now called GoTo. They just merged with the largest e-commerce platform in Indonesia. So across Southeast Asia, we had about 170 million users. Like, it's- Southeast Asia has scale. If you ever wanted to work at scale, you would go to Southeast Asia. We had 20 plus different services from transportation to food, shopping, medicine delivery, bill pay, movie tickets. So it was like all of the startups in America in one app all being built at the same time with the same user base. And so everything was tremendously layered because you could fill all of these opportunity gaps in the market where a single app would probably not be as sustainable. So Go-Jek is massive across Indonesia, Singapore, Thailand, Vietnam. And then Kumu is kind of a super app for social. So Go-Jek was very transactional. It's like, "Here's a job to be done. I wanna pay for something and someone delivers it to me." And with Kumu it's more so of a, "I wanna do Clubhouse, Zoom, Google Hangouts, Gather round all in one app." So we cover social feeds, audio, video, multi seats. There's a ton of different use cases that we serve on Kumu. And Kumu is primarily in the Philippines, but ranks top 10 in a bunch of countries as a top grossing mobile app.

2. 9:40 – 11:31

   How Crystal decided to join a risky startup

   1. LRLenny Rachitsky9:40

      So with Kumu you joined when they're already doing fairly well. With Go-Jek, as you said, you joined very early. What did you see in that company that helped you decide to join such a risky early stage company? For folks that are maybe thinking about joining a startup, what kind of things did you take away at what to look for?

   2. CWCrystal Widjaja9:58

      Honestly, it's probably a lot of luck, but also at that age I realized I have very little to lose. So with Go-Jek, I think I felt like it was the right company because I was able to really clearly understand the value prop. Traffic in Indonesia is crazy. It takes you two hours to go 20 kilometers. So, of course you want to take a motorcycle taxi to beat that traffic. Of course you don't wanna go out and get food and then have to come back this long pathway of two hours. So I think taking that Warren Buffett approach, I knew that the product made sense. The market made sense as well. So drivers there were already a thing, but it was very hard to connect them to the consumer. It was painful to haggle prices. There were lots of restaurants scattered across Indonesia. So the value prop and the market made sense and the channel by which you would do it through this mobile app made a little bit less sense at the time because most drivers didn't have a mobile app. But I felt like it was a problem that was very solvable and we ended up renting a stadium to just hire like 60,000 drivers in a couple of weeks. So I think looking back it was certainly a risk. Like when I got there it was in a house and I realized I've probably made a huge mistake, but we were growing very quickly already even at that small scale of like 4,000 orders per day.

   3. LRLenny Rachitsky11:26

      I wanna spend a lot of time talking about what you learned driving growth at these companies, but one quick question.

3. 11:31 – 13:15

   Why haven't super apps emerged in the U.S.?

   1. LRLenny Rachitsky11:31

      So Go-Jek's kind of the super app where you do a lot of stuff in one app. Do you have any insights into why a super app hasn't emerged in the US?

   2. CWCrystal Widjaja11:40

      Yeah, I think the sentimentality of a conglomerate is very different in Southeast Asia. So we've grown up with, you know, a specific conglomerate owning not just the mall that you go to, but also the apartment building that you live in and the school that you go to. And so they're very well integrated and there's this sense of trust in a conglomerate. Whereas in America we already kind of shy away from like, "Does Google know too much about me?"There's also, I think, the second aspect of it, which is that in Asia, we've kind of leapfrogged on to the computer era. So everyone has a phone, but you may not even have a computer in the entire household. And so when your phone is full, are you gonna delete a photo of your kid, or are you gonna delete this app? You're probably gonna delete the app. So for anyone to really survive, it has to be part of this super app concept.

   3. LRLenny Rachitsky12:34

      Oh, wow, I've never thought of it that way, that you don't have a lot of space on your phone and so you want one app to do a lot of things.

   4. CWCrystal Widjaja12:40

      That's right. So there's a decision factor that you don't really have in the U.S. because the cloud storage and device capacity there is a little bit bigger.

   5. LRLenny Rachitsky12:50

      Hmm. So interesting. So in the U.S. you can have different apps be... Like basically the super app doesn't have to be the best at everything.

   6. CWCrystal Widjaja12:56

      Exactly.

   7. LRLenny Rachitsky12:56

      The fact that it does enough and everything good enough. Wow, fascinating.

   8. CWCrystal Widjaja12:59

      It just needs to get the job done.

   9. LRLenny Rachitsky13:01

      Amazing. Okay. That's super interesting. Okay. So transitioning a bit to growth and things you've learned along the way. So you talked about how I think Go-Jek you said hired like tens of thousands of drivers really quickly.

4. 13:15 – 16:35

   What startups in the U.S. can learn from companies in Asia

   1. LRLenny Rachitsky13:15

      Are there things that startups in Asia do that you think companies in the U.S. should do and can learn from in terms of growth?

   2. CWCrystal Widjaja13:22

      Yeah. So we did kind of crazy things, right? Like i- if someone told you in the U.S. that they were gonna rent out a stadium, preload a bunch of mobile devices, market that drivers should come here en masse for a job fair, they're gonna give them a phone and send them on their way, like some people would kind of say no. Like, "That's kind of crazy. Won't we get in trouble?" And to an extent maybe that's true, so maybe there are some limitations there. But this concept of doing things that are somewhat crazy but validate a point, doing stuff that don't scale especially, I think is really the bread and butter of what we did at Go-Jek. Like, we were insanely scrappy. We would do things as simple as wanting to test s- a subscription feature, which was just released in Singapore couple of weeks ago. We ended up saying we have this voucher system that we can distribute vouchers in the back end. We obviously know our drivers' phone numbers. Why don't we just add them to a WhatsApp group? We'll add a hundred drivers randomly to a WhatsApp group. We'll tell them, "Every time you are on a ride with a customer, try to sell them this pitch. You are the only driver who can sell, you know, a subscription package. Have the customer give you $10. Text us when they say yes. Someone will be sitting by this phone all day every day. We'll look up the customer that you were on a ride with in the back end. We'll give them the vouchers in the back end, and then we'll deduct $10 from your balance." Like, it works. It's really this Wizard of Oz experience. We don't have to build anything. I coordinated with a bunch of interns, and we were able to validate some of the value prop and conversion rates that we would expect in a subscription service. When we wanted to do a new onboarding screen, but turns out we have lots of engineering work to do, we took a screenshot of the screen as is and we just had our designer put what the onboarding flow might look like if we had to overlay it on top of the screen, and we just sent that as, like, an in-app message. And then eventually I think finding stuff that does scale intuitively. We knew that we were sending out lots of fake features through things like Typeform surveys, things like a personality quiz can be very easily done through Typeform. And we realized that if we built in the in-app web page and we made it easier for us to do a website deployment on our back end side, we wouldn't have to wait for a mobile app release to test some of these new features out that could be done on web. So it's really just, like, what is the, the user experience that we wanna create? How do we manifest that as quickly as possible? Let's just try that first.

   3. LRLenny Rachitsky16:14

      Going back to the stadium example, I knew you said that you hired a stadium full of people. I didn't realize it was actually a stadium that you-

   4. CWCrystal Widjaja16:19

      It was literally a stadium that we rented, like a football field, a couple football fields if I'm not wrong. There's long lines, boxes of phones and SIM cards. So, it was a lot of just, like, doing really hard work to get to that scale.

5. 16:35 – 18:28

   How to get intentionally scrappier in your organization

   1. CWCrystal Widjaja16:35

   2. LRLenny Rachitsky16:35

      Wow. I know you do a lot of advising too. Do you advise startups to be more scrappy and do things that don't scale? I imagine 'cause in the U.S. the culture is a little different. Mm-hmm.

   3. CWCrystal Widjaja16:45

      Like, the only thing better than knowing... Like, if you have data of what your customers are doing, that is the best data you could ever get. And so if you don't have a tested hypothesis, if you can't think of a way to run an experiment, then honestly that idea is pretty useless. Like, maybe it makes sense to throw the market, to the model. But you could have weird consumer sentiments. Like, not everyone is a rational actor. So testing the actual experience and seeing how people respond to it, that's the best possible data.

   4. LRLenny Rachitsky17:21

      Pulling that thread a little bit, for startups, experiments are often hard 'cause there's just not enough data, not enough users. How do you think startups s- should approach that? Can you run experiments when you're really, really early?

   5. CWCrystal Widjaja17:32

      You should. I mean, even if you have a sample size of 30, the data you get back generally does not change but its precision will. So mathematically speaking, you're gonna get the same level of trends, but it... The precision at which you understand those trends will become more deep if you have more data. But the o- underlying information that you're getting out of that won't be very different at larger scales. So what's better than having 30 data points? Certainly having a hundred, but what's better than having zero is definitely 30.

   6. LRLenny Rachitsky18:10

      Fascinating. So contrarian. Running experiments at 30 people. I love that.

   7. CWCrystal Widjaja18:14

      You have to. I mean, like, every idea is so cheap at that scale. Like, you could do things that don't scale dramatically better with 30 people than at 100, if you're testing.

6. 18:28 – 21:26

   How to get and utilize scrappy and small data sets

   1. LRLenny Rachitsky18:29

      So when you s- just to kind of pull out a little bit, when you're running an experiment with 30 people, what do you look for? You're looking for, like, 20 of them to do something, like a large percentage of that group does something, right?

   2. CWCrystal Widjaja18:39

      So everyone wants to go on, like, retention, they wanna see that users are doing this thing, and they wanna get from step zero to 100 really quickly, but they don't realize that, like, users make decisions based on succeeding events. So what's one step before the user makes that decision? What are the things that they have to do, the things that have to be done? So we're always looking for what is a specific reason that this user might have converted. For things like GoFood, it would be things like when does the user try a new merchant if what people are e- ordering right now are just food that they already trust and know? If you need to have trust in order to purchase food from a merchant, how do we generate that trust? So we actually hacked it by connecting people's Facebook connect login. So we had already had permission to look at who they had connected with on Facebook. We actually looked at the food that their friends had purchased and used that as a dataset of, "Hey, here's food that Lenne purchased and liked. Maybe you would like it too." And so that was one way to hack the trust factor. And we did find that when we told people, "This friend purchased from this merchant," you would be twice as likely to purchase from a brand new restaurant than users who did not have this feature. And that increases GMV that eventually gets you to the conversion rate that you wanted, but it solved a different problem before. How do I convert? It was, how do I solve for trust? How do I break the barrier of facilitating that decision-making process, that aha moment by fixing the set-up moment, which was trust.

   3. LRLenny Rachitsky20:31

      And that's just a general kind of rule of thumb you have. Don't use retention as a goal. I know you wrote about this somewhere. Is that kind of a rough, rough rule of thumb you use?

   4. CWCrystal Widjaja20:40

      I think a lot of people thought that I had meant, like, retention sucks, don't care about it at all. But in reality, it was really, like, when you think about retention, like, that's just not specific enough. So there is this, um, mental model that I use from Made to Stick where they'll tell you, like, "Lenne, think of everything in the world that is orange." And you're like, "An orange. What else?" And then if you change that structure, that sandbox, to think of everything orange that's in a construction site, then you really start to realize and grasp at concrete concepts and can actually action on them in real life.

   5. LRLenny Rachitsky21:25

      Got it.

7. 21:26 – 22:28

   How to increase retention

   1. LRLenny Rachitsky21:26

      Speaking of retention, where have you found products and companies have the most success increasing retention?

   2. CWCrystal Widjaja21:32

      It's usually in the step right before conversion. So if they aren't sure, like, why the user opens the app, or they aren't sure why the user got to this checkout page, it's often some, like, copy or the path has been ineffective in some way. I'd like to see founders think about the user psych model that Darius Contractor often talks about. So you need some momentum in that user journey to get them over the hump of some of these very painful user processes, like typing in a credit card. That's a lot of work. How do you lower that friction? And being able to sequence the right steps effectively and just moving around screens actually can do a lot.

   3. LRLenny Rachitsky22:21

      Going even deeper there, so the companies you've worked at, the companies you've advised, you're on the boards of a couple companies, I noticed.

8. 22:28 – 25:01

   What does and doesn't work in growth?

   1. LRLenny Rachitsky22:28

      What have you found to be really good uses of time in terms of growth investments, like things that often work? And then kind of a second question, what do you find is rarely successful where people invest a lot of time and ends up not being really useful for growth?

   2. CWCrystal Widjaja22:44

      Yeah, I think I see a lot of founders, like, grasping at straws. So there'll be, like, this brand new feature that does something kind of different from what people are already doing on our app, like this will make things work. But they don't have any Wizard of Oz test, they d- haven't proven that people want to do that. They don't have any data of users currently trying to do that. And that's a, that's a sign of, like, why this instead of literally anything else that you could be doing? I do find if you have a lot of people landing on a webpage or an app and then not doing anything, then it's probably copy. Like, they haven't even experienced the product. It's clearly not the product that's wrong. So how can you change the copy and resonate with the pain point rather than the solution you are offering so that users understand how to fit themselves into the use case? So copy is a big one if I see conversion rates aren't landing between app launch to some first action. But if there is conversion and they're just not as frequent, I try to look at what the most painfully long conversion events are, so users who eventually check out or eventually completed the aha moment. What are the user paths and what is the longest one that seems like it's the most painful?... are there enough people trying to do that? And how do we shorten that cycle? So for Kumu, it was things like, users wanted to sign up and find their friends on Kumu. And so, they were using search. Frequently, search was under-utilized API, it was kind of slow. We sped that up. Conversion rates go from 60% to 90%, like, over the course of a few weeks of just optimizing that and putting more content there. So, looking at, like, where are people doing things and then failing? Like, you already know this percent of people would convert if you fixed this. That's a definite potential win. So we try to layer these definite wins with, like, crazy bets of, like, brand new feature with no data. At least run an experiment if you can. But I always try to layer in these sure wins.

9. 25:01 – 28:03

   The conversion rates Crytsal looks for to determine viability and success

   1. CWCrystal Widjaja25:01

   2. LRLenny Rachitsky25:01

      When you talk about conversion being good and bad, do you have a rule of thumb or kind of a mental model of, like, here's a rough range of, like, this is good and we should not really spend all our time on this, and this is bad and we should optimize?

   3. CWCrystal Widjaja25:13

      So, assuming that the frequency is correct, so you have a weekly frequency. If users are coming back, if it's a free product, 60%. Right? It has to be at least 60% if it's a free product week over week. If it's a paid product, I usually look at that more as, like, maybe 20 to 30%.

   4. LRLenny Rachitsky25:31

      And this is retention, people coming back the next week?

   5. CWCrystal Widjaja25:34

      Exactly. Coming back in the second week or month, or whenever your frequency ratio is. And this is at scale. So if you are much smaller, like your friends and family, that better be near close to 80% no matter what. Because if you can't even convince the people who care about you to use the product, it probably isn't gonna solve the job for anyone else.

   6. LRLenny Rachitsky25:56

      Very handy. Very concrete numbers. And then your point is that when you're a startup, it's only gonna go down because your kind of early adopters are more excited and, and they'll be more excited about coming back. And so, yeah, so you wanna start really high.

   7. CWCrystal Widjaja26:09

      I mean, don't make the same mistake that, uh, Netflix and Spotify have made, which I guess is when they've launched, they've started international expansion, and they see this very small percentage of users start to sign up for Spotify or Netflix. There are very few people though in Southeast Asia or internationally that have the types of credit cards that Spotify or Netflix would accept. And so, when they launch in these markets and they see a ton of uptick in the first week, they're like, "This is only gonna get better." When in reality, it's like, you just pulled forward everyone who could have possibly subscribed to you. Now you're gonna have to work a lot harder to get everyone else.

   8. LRLenny Rachitsky26:48

      The 60% number, um, so you're th- you're saying it's like, then every week, 60% of the previous week come back roughly as just a rule of thumb?

   9. CWCrystal Widjaja26:54

      Yep. Exactly.

   10. LRLenny Rachitsky26:55

       Is that kinda how you think about it versus, say, cohort retention? Is that just 'cause it's easier as just a simple rule of thumb?

   11. CWCrystal Widjaja27:01

       I am actually thinking of it as cohorts. So, 60% should be your week one, and then it should flatten. I think I usually give teams, like, two to three weeks or frequency periods to see things flatten. But it better flatten around 60% for a free product. That's kind of, that's actually what we saw at Gojek. Early days, it was, like, 60, 70% retention rates because people were using this product that really solved a huge problem for them. And I think that's when I knew we were gonna be fine. If people keep coming back, the product just needs to work.

   12. LRLenny Rachitsky27:37

       Wow. So, week one, 40% of people drop off. Week two and beyond, basically nobody drops off is kinda what you look for.

   13. CWCrystal Widjaja27:43

       Yep.

   14. LRLenny Rachitsky27:43

       Wow. What a, what a high bar. But I like that, 'cause-

   15. CWCrystal Widjaja27:47

       Yeah. Well, Gojek is a decacorn.

   16. LRLenny Rachitsky27:49

       Mm. Okay, there we go. (laughs) If you wanna be a decacorn, there's your new benchmark.

   17. CWCrystal Widjaja27:55

       Exactly.

   18. LRLenny Rachitsky27:55

       Amazing. Okay. There's a bunch of other stuff I wanna dig into. One is, is just data modeling and thinking about growth strategy as a

10. 28:03 – 32:44

    How founders should think about approaching growth

    1. LRLenny Rachitsky28:03

       founder. So, say a startup is just trying to think about, "How do we drive growth? Where do we invest?" Do you have kind of a framework or a process? I know this might be a really big question, but just for founders to think about, to think about how their growth works, what their drivers might be. How would a founder approach that problem?

    2. CWCrystal Widjaja28:21

       For sure. So, I thought that this was not an obvious process. It wasn't like an explicit process, until I worked with Reforge to build my Data for PMs program. Gotta get that plug there.

    3. LRLenny Rachitsky28:34

       Go Reforge.

    4. CWCrystal Widjaja28:34

       But I basically talked with the Reforge folks about like, "Here's what I would do in all of these scenarios." And they're like, "Oh! So you mean you're doing this step one, step two?" And I was like, "Yes, actually. How did you figure that out?" So I don't really think in frameworks. Like, this is just a logical process to me, but I think what I've figured out is it's step one, like, you have constraints, right? Similar to our sandbox example of, like, everything in the world that's orange versus everything in a construction site. You have to think about the physics of the current market, the product, the model, and the channels that you're using. So to use Gojek as an example, it would be market of Indonesia. Here are the consumers in this market, the driver's side, supply side in this market. Here is the product, mobile app. We're able to connect drivers and consumers. There is a allocation that we create. Model: we charge per order, channel: we are able to do this through push notifications or in acquiring new users, it might be through Facebook ads or ... and this was a really big insight for us, it's the real world. There is a physical conception of a driver in a jacket driving around the city who is marketing Gojek for us. And word of mouth actually was primarily driven by, "I saw a driver on the street, so I knew Gojek was here." And that actually was a huge driver of all of Gojek's growth as it expanded to new cities. So step one is, what are the physics? Step two is, when you think about loops and growth funnels and the quantitative inputs to each loop...... does that fit into these physics, or do you have to change, like, four or five different things? So, we were very careful about changing too many parameters and making too many bets on too many variables going our way. So we would always change, like, one small thing at a time and make sure that it fit into the model.

    5. LRLenny Rachitsky30:38

       This episode is brought to you by Eppo. Eppo is a next generation AB testing platform built by Airbnb alums for modern growth teams. Companies like Netlify, Contentful, and Cameo rely on Eppo to power their experiments. Wherever you work, running experiments is increasingly essential, but there are no commercial tools that integrate with a modern growth team stack. This leads to wasted time building internal tools or trying to run your experiments through a clunky marketing tool. When I was at Airbnb, one of the things that I loved about our experimentation platform was being able to easily slice results by device, by country, and by user stage. Eppo does all that and more, delivering results quickly, avoiding annoying prolonged analytic cycles, and helping you easily get to the root cause of any issue you discover. Eppo lets you go beyond basic click-through metrics and instead use your North Star metrics like activation, retention, subscriptions, and payments. And Eppo supports tests on the front end, the back end, email marketing, and even machine learning clients. Check out Eppo at GetEppo.com, GetE-P-P-O.com and 10X your experiment velocity. So step one, just to cover this, is figure out how you're growing. In Gojek's case, it was partly real world, people just seeing, seeing Gojek riding around.

    6. CWCrystal Widjaja31:58

       I think it's, it's both figure out how you're growing and also the elements that you have at your disposal, like what are the levers that you have that maybe you've never tried using? When we looked at our model this way, we actually realized we had under-utilized the driver's capacity to drive our growth, pun definitely intended. So-

    7. LRLenny Rachitsky32:19

       (laughs)

    8. CWCrystal Widjaja32:21

       ... in looking at the model this way, we had thought through, like, "What is our goal?" We want Gopay to be much bigger than it really is. It's a e-wallet service. Users are able to get access to this digital balance. How do we drive adoption? And so when we looked at the lever of we have a driver, we actually created an incentive

11. 32:44 – 34:53

    Figuring out how to grow and the resources you have at your disposal

    1. CWCrystal Widjaja32:44

       model. So, we built a very small service that would check when a driver got allocated to a customer, again, the product and the model. We would then check in the database, has this customer ever used our Gopay product before? Did they have a digital balance? And if the answer was no, we would message the driver immediately, "Hey, this customer hasn't done a Gopay top up before. If you get them to give you cash, and we deposit it into their virtual wallet, we'll give you extra money." So, using them as the salesperson, you wouldn't believe how great of a salesperson someone can be when you are literally trapped in a car with them (laughs) going somewhere. And so you have this captive audience, captive attention. You have someone who has the incentive to cross-pay or cross-sell someone into Gopay. And customers were able to feel the benefit because the driver was explaining it to them directly. So, it was one small... There was no change to the physics. It was a lever usage.

    2. LRLenny Rachitsky33:49

       What a devious strategy.

    3. CWCrystal Widjaja33:50

       It was huge. It was like 60% of acquisition once we released that.

    4. LRLenny Rachitsky33:54

       Oh my god. So for thinking through your potential levers and physics of your growth, do you think about it like bottoms up? Here's all the things that are going on and here's areas we can invest. Or do you have kind of like a menu of options top down of like, "Here's the 10 things that could be." It's looking like for Jo- Gojek, it's these four, and let's focus on that.

    5. CWCrystal Widjaja34:16

       Yeah. I think you, you always have to start from the fact that like, we are not wizards. Like, it's very hard to move the physics of a universe when you are trying these new things. So, start with what currently works and currently exists and where you think the biggest constraint is or the best lever is, and then fix that one piece. Because the entire universe isn't exploding. Like, you're not... The world isn't changing so dramatically that your physics change. So I think rooted in reality is very important.

    6. LRLenny Rachitsky34:50

       Got it. Okay, so it's see what's working, find the constraint,

12. 34:53 – 37:40

    Optimizing the funnel to make growth happen faster

    1. LRLenny Rachitsky34:53

       and then step two is basically what can you do to the product to optimize the funnel/loop to make it go even faster.

    2. CWCrystal Widjaja35:00

       Exactly.

    3. LRLenny Rachitsky35:01

       Love that. Maybe, um, as another example, if something comes to mind with Kumu, is- how do you think of Kumu through this lens?

    4. CWCrystal Widjaja35:08

       Yeah, I'm always very hesitant to talk about Kumu 'cause there's so much competition right now and we are-

    5. LRLenny Rachitsky35:13

       Okay.

    6. CWCrystal Widjaja35:13

       ... like on the cusp of some very interesting, interesting things. But I think for Kumu, it's actually very complex because there's a lot of human emotion that is involved. Like, with Gojek, you knew if you got the job done, you made a transaction. With Kumu, how do you know if a consumer made a friend, like, felt like they had a genuine friendship? So you almost have to create more friction to identify users who really got past that barrier and are explicit with the activity that they did. So we have features that tell us if a user is really searching for this job to be done. If they really wanna be part of a community, how do they fill out this? Do they fill out the form? Do they fill out a questionnaire of like many questions? Do they go through this friction just to get access to a community? So we almost create this artificial friction to help differentiate how deeply a user wants something or needs something. And if a user doesn't fill out that questionnaire, maybe they're actually looking for something else. They were looking for entertainment, they were looking for content or short form content. And so creating...... almost, like, hand raiser, uh, approaches for a user to say, like, "I wanted this thing." We leave a lot of breadcrumbs in the app to be able to identify those paths.

    7. LRLenny Rachitsky36:38

       Awesome. While we're on the topic of these two companies, just maybe for inspiration to founders who are thinking of ways to drive growth, what were a couple of the bigger unlocks growth-wise for these two companies or even any other company that you've worked with that's interesting?

    8. CWCrystal Widjaja36:52

       Yeah. Definitely in the early days, it was copy. So I think if your product does something that's not super familiar, you have to tie it to something that is. So I talked about using drivers to sell GoPay. Before that, one thing that we did was to actually take someone's virtual account number and put it onto a picture of a credit card. Like, you know what a credit card is. That's familiar to you. A lot of people didn't know what a digital wallet was. And so when they looked at this, like, "Oh, okay, I have this virtual thing that acts like a credit card. It works like my debit account," then they understood the concept a lot better. And we actually saw top-ups increase based on us literally just sending that picture with someone's virtual account

13. 37:40 – 40:07

    Crystal's biggest lessons on unlocking growth

    1. CWCrystal Widjaja37:40

       number there. So they could go to an ATM, and they would just type in the card number as they would a regular debit account, and they realized that they could top up through that channel. So that was something that was pretty interesting to us with just, how do we tie the familiarity loop back into the consumer mental model of the product and drive acquisition that way?

    2. LRLenny Rachitsky38:02

       And that was at Go-Jek?

    3. CWCrystal Widjaja38:04

       Yep.

    4. LRLenny Rachitsky38:04

       Is there anything else, maybe, since you don't want to talk too much about Qumu, any other, like, advisorships or companies, examples of something that ended up working really well to help them accelerate growth?

    5. CWCrystal Widjaja38:14

       Things that have worked really well. So for one of the companies I work with, AB InBev, they run a lot of their D2C brands in South America and globally. So one of the features that we were looking at was, how do we ensure that subscriptions don't actually become a canceling point for a user? So in the app, you could cancel or you could resume your subscription, but you couldn't pause it. So when we looked at the cancelation reasons and we saw that their number one reason was, "I still have too much beer," we actually decided, "Well, let's just add a pause button then," 'cause, like, canceling the subscription is a permanent solution to having too much beer. How do you make a temporary solution that solves the actual problem? Adding in a pause button actually helped alleviate a lot of the churn that was becoming very hard to re- re-acquire back. So that was one fix where we looked at the, again, physics of the model. Like, we're not gonna create new changes to the product or create one-time buys or, like, reactivation emails. We'll just solve the problem at that small constraint where everyone drops off.

    6. LRLenny Rachitsky39:35

       Wait, so can you order beer subscription? Is that a thing? Is this a consumer product, or is this, like-

    7. CWCrystal Widjaja39:39

       This is.

    8. NANarrator39:40

       Yeah.

    9. LRLenny Rachitsky39:41

       (laughs) Cool. Okay. This also reminds me, at Airbnb, this was actually one of the biggest wins, is adding a snooze feature to your listing. Exactly-

    10. CWCrystal Widjaja39:50

        Exactly.

    11. LRLenny Rachitsky39:50

        ... the same thing. Yeah. All right, there we go. Awesome tip for folks that have churn problems, snooze/pause. I wanna shift a little bit to a post that you wrote that maybe is one of your more popular posts you wrote on the Reforge blog called Why Most Analytics Efforts Fail.

14. 40:07 – 44:24

    Why most analytics efforts fail and how to avoid failure

    1. LRLenny Rachitsky40:07

       And-

    2. CWCrystal Widjaja40:07

       Yes.

    3. LRLenny Rachitsky40:08

       ... I'd love to hear your broad overview of why do most analytics efforts fail? And then how do, how do teams avoid this? Maybe what are, like, two or four things they could, two to three things they can do?

    4. CWCrystal Widjaja40:18

       Yeah, I'm actually pretty surprised at how much noise that has generated, because I guess it came from a place of frustration, where I kept telling people, like, "You're doing this wrong. Here's how you should probably be doing it." But I think it resonated a lot with folks, because they recognized all of those symptoms, but they weren't sure why it was happening. So to say, like, "Oh, this is the thing, instrumentation is what's wrong," I think it's a very actionable thing. Like, it's probably one of the most solvable problems out there. It just takes some time and mental model shifts to do it well. So a lot of people look at tracking data as, "How do I track my OKR? How do I know if I'm going up or down?" But they don't use it to track or identify insights. So I will use the example of using Twitter for, quote unquote, "news," when in reality, they're actually using Twitter for entertainment. Do not treat metric gathering as entertainment. Like, it's not there for you to be, like, "Oh, that's interesting. How novel," and then not act on it. So real news is information that changes what you do in the real world. And if you don't change what you're doing, what you are doing is just getting entertainment. So let's use that as the premise. The next step in instrumentation is to look at the fact that measurements do not equate to insights. A measurement would be an observation. It's a data point in your database. So the example being power users do four times more bookings is an ob- observed fact, because your transactional database obviously says that that is the case. But it's not an insight 'cause it doesn't have context. It doesn't give you information that lets you act on it and better understand the problem. So another example would be if I see my girlfriend hanging out with a guy I don't know, that is an observed fact that you see in the real world. Your hypothesis could be that your girlfriend is cheating on you. But the insight, uh, the actual fact might be that she's not cheating on you, it's her cousin.And now your insight is, I am paranoid, and I need to change my behavior to be less crazy. So the insight will provide value when you have this why answered. Why is this person doing this thing? Here's why. And then you are gonna act differently. So for our purposes, if we look at a GoFood user will transact and is more likely to use a voucher, that's a fact, that's an ob- observation, but it's not an insight. An insight would be something like, GoFood users who are power users are more likely to use a free shipping discount on a high GMV basket, versus non-power users. And that actually tells you how to change your marketing approach. It tells you that in what circumstances does someone do this? When it's a high GMV basket, give power users the ability to get a free discount, but do not do this for non-power users, 'cause they won't convert any better than they normally would. So that helps you change your marketing spend. It helps you understand the decision points of power users versus non-power users. The insight is instrumenting properties into an event so that you can segment who is doing what behavior, and make some hypo- hypotheses on that observation, test that hypothesis, and then you get some causal representation of whether or not that hypothesis was right.

    5. LRLenny Rachitsky44:10

       So it sounds like a lot of the root of the issue is setting up the wrong metrics, the wrong, I guess, there's the tracking, uh, element of just capturing the right information, and then also just not focusing on insights versus just having a bunch of information.

15. 44:24 – 46:17

    Signs your organization is getting analytics wrong

    1. LRLenny Rachitsky44:24

    2. CWCrystal Widjaja44:24

       Exactly.

    3. LRLenny Rachitsky44:25

       What are signs that you're doing this? Like, say someone's gonna go load up their dashboard and they're like, "Am I, am I failing or not?" What should they be looking for?

    4. CWCrystal Widjaja44:33

       So, I already know if s- if a team is good at instrumentation or not just by looking at the instrumentation spec. The symptom of a bad data tracking approach is, you have a ton of rows with a ton of events, but every event has, like, one property or no property being tracked. So an example with Gojek would be when a user lands on the map to select a drop-off point, the event would be drop-off, or, like, map loaded, let's say. And the properties there should be things like, how many drivers do they see on the screen? What is the pickup location? Is it... What city is it in? What latitude and longitude is it? What is the s- is there surge pricing? What is the current minimum fare? Do they have a voucher code? All of these characteristics of the experience and the context that can help you look at, hey, when a user only sees two drivers on the screen, they're much less likely to convert than a user who sees five drivers on a screen. Now we can look at, in what cities and in what latitude and longitudes do we mostly only see two drivers versus five drivers? Like, being able to do this second layer approach of the why and not just stop at, hmm, that's weird, when you have two drivers, you are less likely to book, but then you never ask why. Like, that drives me crazy. Or the inability to even know that, like, there were only two drivers on the screen. Like, you're missing so much context of the user's experience that you're unable to make assumptions about why the user didn't convert.

    5. LRLenny Rachitsky46:15

       I love this. Is there,

16. 46:17 – 47:05

    The best resources for figuring out how to do analytics right

    1. LRLenny Rachitsky46:17

       is there a resource, like, maybe your course is probably gonna be the answer, but for folks that want to figure out how to do this sort of taxonomy and events well, how do they go about doing that?

    2. CWCrystal Widjaja46:26

       So I think it's important to just, like, go through examples. Yes, every product is different, but everyone has the same sign-up flow for the most part. So look at the sign-up flow examples that I have in the blog post, or in, I believe, Amplitude actually has a pretty good long-winded documentation on this, on how to do event tracking. But it's really a matter of, like, sitting down and thinking really deeply, if I were to press this button, why would I and why would I not? And am I tracking that in my, in my user properties? So it's really just like sitting down and mapping out the experience.

17. 47:05 – 48:37

    Crystal's recommendations for metric tracking stacks.

    1. CWCrystal Widjaja47:05

    2. LRLenny Rachitsky47:05

       Speaking of Amplitude and other data tools, do you have a default recommended metric stack for founders just to start with and maybe a few other things as they evolve?

    3. CWCrystal Widjaja47:14

       It really depends on how early they are. So if they have a single data warehouse with all of their transactional data, usually I say, like, you can probably get by with Google Data Studio, it's free, usually with whatever you're using. If not, Metabase has a great open source free tool. If you have someone who can write SQL or if you have multiple databases, then Metabase is great. If you need an app mobile device event tracking, I usually recommend Clevertap because Mixpanel has unfortunately failed me a lot, and Amplitude doesn't have the CRM components that I would need all in one space. If I am much bigger and I need more analytics juice, maybe Amplitude makes sense on top of this, or something like that helps me pipe data into more dashboards and do less ETL for me, then I would get into Segment. And then once you get into experimentation, obviously have to shout out to Epbo. I think they've really re-instrumented a lot of the dashboards that I would have normally had to do in ex- in experimentation projects. So I usually look at something like Epbo to just automate the decision-making flow.

    4. LRLenny Rachitsky48:21

       Awesome. I think we're both small investors in Epbo, but, uh... Big fans, but a little biased, but yeah, it's an ex- Airbnb team that built it, so it's cool. Shifting a bit from metrics and data to just growth teams in general-Maybe first question

18. 48:37 – 51:38

    How you should set up your growth team originally

    1. LRLenny Rachitsky48:37

       is just how do you recommend companies set up a growth team in the early days, and then o- over time?

    2. CWCrystal Widjaja48:44

       Yeah. So I can talk about how growth was set up at Gojek as an example, which I think is probably the best practice. So we didn't really know that- what growth was at that time, but we knew there were obvious gaps to fill. So because we had grown so quickly, the core product team was still making the core product features, like as simple as, like, phone number masking. That wasn't a thing yet. Like, you had access to your driver's phone number, it's probably not a great thing, it's probably part of the core functionality and we need to fill that gap. At the same time, growth was still necessary because you have all of these users trying to use a product that aren't quite getting there. So things like figuring out what SMS provider we should use to send the OTP to this user who is signing up from this telco provider. That was a growth objective that, like, isn't necessarily core feature work, but was a gap to fill given the onboarding and SMS success delivery rates. Things like telling the driver if this was a brand new customer, 'cause at this point in time, drivers had taken thousands o- of rides and they assumed every single customer knew how Gojek worked when maybe they didn't. And so the- we knew that the protocol was that a power user would know, they would make an order and they would just wait. They would wait somewhere, they would keep an eye out for a driver, and then they would get on the motorcycle and go. But for a brand new user, are you supposed to walk to the driver? Like, are you supposed to find them? It's unclear to this brand new, uneducated new user how to use the product. And so first time user experience could have been a terrible one, where they went and walked off and then the driver came to the pick-up point and they couldn't find them. So it was all of these, like, small acquisition, adoption and engagement use cases that growth was filling the gap on, and eventually we embedded our growth, I would say product managers at the time, into these teams and they ended up kind of synthesizing what growth was as a full-time role, eventually becoming PMs who owned specific parts of the product stack.

    3. LRLenny Rachitsky50:58

       So in your experience, and I hear this a lot is, your first growth person shouldn't just come in and figure out what to work on. You should understand, here's where we need growth help, let's find somebody to tackle it, versus come help us figure out what to do to drive growth. Is that how you've seen it?

    4. CWCrystal Widjaja51:12

       Exactly. I think it, it's just setting the bar, like, too high to expect someone to come in and model everything. Like, again, there are physics in place, it's very hard to move everything, so it's really about having someone who already has all of this data, knows where the biggest gaps are, doesn't have to start from scratch and figure this out, and then just picks some small space to work on that they know is workable.

19. 51:38 – 52:43

    Integrating growth teams or separating them?

    1. CWCrystal Widjaja51:38

    2. LRLenny Rachitsky51:38

       Do you have strong opinions about growth being integrated the way that you described, where a growth PM is basically, has a cross-functional team basically as the PM, versus kind of a separate growth team that's off to the side?

    3. CWCrystal Widjaja51:50

       Yeah. I think it can work as a separate growth team to the side if the company is truly, like, head over heels tripping on insane product market fit. Like, if there is insane product market fit and you are really scrambling to do core feature stacks, then maybe a growth team to come and be clean up is fine. Like, we really called ourselves, like, we're the clean up crew. We pick up the pieces that were left behind, we connect the dots, like, you forgot to plug this in, we'll plug it in for you. But we were a team of, like, lots of stats heavy people, so a lot of my team were, like, statistics graduates, we cared a lot about looking at numbers and odds and probabilities, because it really is a numbers game at that scale. You could work on anything and everything would probably do something, but what was the thing that would make the most impact now and unlock us for the future?

20. 52:43 – 53:55

    Who should be the first growth hire?

    1. CWCrystal Widjaja52:43

    2. LRLenny Rachitsky52:43

       I was gonna ask you what to look for when they're hiring an early growth person. Is that what you find, just stats data kind of person?

    3. CWCrystal Widjaja52:49

       Yep. You, you have to have someone who knows how to run the numbers, right? If you're looking at ratios of conversion rates but you don't realize that this ratio is of a b- much smaller base size, you're gonna make the wrong decision. So someone who is intuitively good at statistics, they know how to do sampling appropriately, they know what selection biases. Like, the worst possible thing is to have a growth person who thinks they are doing the right thing and is measuring things wrong and then focusing on the wrong areas.

    4. LRLenny Rachitsky53:24

       Do you find that it's often easier or better to hire a young up and coming person or find someone that's got a bunch of experience for your first growth hire?

    5. CWCrystal Widjaja53:34

       I would hire someone who is willing to take an intro to statistics course. And it doesn't matter, like, if they've had the experience, like, go wild or not. I think it really is, like, can they focus on the right opportunity rather than the most flashy thing? And I think both profiles can come under that.

21. 53:55 – 56:55

    How to hire a great growth person for your organization

    1. LRLenny Rachitsky53:55

       Got it. And then what do you do in a hiring process for someone like this? What kind of things do you suggest founders look for?

    2. CWCrystal Widjaja54:01

       Yep. I actually look for that first principle bias. So I'll give people case studies of, like, "Here's what we see. How do you know that this is true?" And then I have them set up an experiment design. I wanna see that they are sampling randomly, not that they're like, "I'm gonna build this feature and launch it and of course it's gonna work." I wanna see that they are taking a measured, deliberate approach to considering, like, why someone might do this or what tools are available. So a growth team can go terribly wrong when they just try to, like, onboard a bunch of brand new tools that don't integrate well and it takes six months to integrate fully.... and then they get nothing done for six months. Like, everything in growth is an opportunity cost of time, trade-offed with what you could have been doing to the product in that time. So we bias towards, like, really quick, hacky things. Like, in the early days of Gojek growth, I think our, our first real growth experiment, we were actually still the data team at this time, was to connect a quick Python script to the Twilio API that we had access to, and we SMSd a bunch of drivers through a CSV that we uploaded that said, like, "Hey, your, uh, acceptance rate is really low. You're not supposed to do that. Please accept all the rides that you are getting." And that actually increased acceptance rates by 2% across the board. And, uh, when we looked deeper into that data, it did, um, even more so for brand new drivers. And so we then worked with the data- uh, driver onboarding team so they could better facilitate the onboarding experience for the drivers.

    3. LRLenny Rachitsky55:38

       For the interview question that you described, like an experiment design question, do you give that as a, as a project where they kind of have time to work on it, or is it a live thing?

    4. CWCrystal Widjaja55:46

       Yes.

    5. LRLenny Rachitsky55:46

       Okay.

    6. CWCrystal Widjaja55:47

       Yeah. I don't think live works really well, uh, for these case studies. Like, I want to see people put in the time and the work to do something to the best of their ability. And, of course, we ask them, like, "Hey, you have five days. We expect you to spend probably, like, four hours on this. So if you don't have four hours within these five days, let us know." So we're pretty careful about giving them the appropriate amount of time to do it at the level of quality that we would have expected if they were to work here full-time. So give them those four hours. We want to see, like, do they Google? If they can't figure it out right now, like, let's see them Google it. We'll ask them, like, what approaches they took, how did they figure this out? And we like to hear people say that they literally had to Google this and read a bunch of white papers. Like, I do that as well.

    7. LRLenny Rachitsky56:39

       For people trying to design one of these for themselves, do you have a question that you've retired that you could share, or something that would help somebody design their own kind of prompt?

    8. CWCrystal Widjaja56:49

       Yeah. I can give you a template after this call.

    9. LRLenny Rachitsky56:51

       Amazing. We'll include that in the show notes. Easy-peasy.

22. 56:55 – 1:02:07

    How to help women and young girls get into STEM (a non-profit that you started)

    1. LRLenny Rachitsky56:55

       Amazing. Okay. (laughs) A last topic that I wanted to cover is a very cool thing that you're involved in. It's a nonprofit that you started called Generation Girl, and I think the mission is to help women and young girls get into STEM. So I'd love to hear about this program, how you got into it, what it's all about, and then also just how listeners can help support what you're doing.

    2. CWCrystal Widjaja57:15

       Absolutely, yes. Generation Girl is very near and dear to my heart. So I co-founded this with a couple of amazing women who were also at Gojek, but are now full-time at Generation Girl. So this really stemmed from us repeatedly getting annoying comments about working in STEM. So things like, "You can't possibly be the engineer on this project." Like, "You look like you like makeup and stuff." And we were like, "Yes, I absolutely love makeup, but I also am badass at writing Swift code. So, step aside." So having experienced a lot of the kind of misrepresentation of what an engineer should look like or should like, I think we really look to, um... Legally Blonde is one of my favorite movies that represents you can take the powers that you have, whether you like engineering or design or data, and you can be whoever you want and still kick ass at it. So, a lot of the women that we support were actually happy if they go into one of our classes and they say, "Actually, I don't like engineering." That's great. That's agency and empowerment that they got to make that decision for themselves without any cultural biases or social pressure telling them that they should feel this way. So we offer free classes for girls 12 to 17. We have college classes. We partner with teachers about how to teach STEM topics, especially in areas where they don't have laptops for every student. Like, how do you teach how to use Figma and things like that? So people can definitely support us and reach out to us. We have a PayPal on our website. Take a look.

    3. LRLenny Rachitsky58:59

       Can you share some of the impact that you've seen from this? What... Are there numbers you can share, anything that you can share around what, what the organizations have done?

    4. CWCrystal Widjaja59:05

       So we've already had, uh, several thousand students go through Generation Girl summer clubs and programs and classes. So we have an event every week. We have a full summer club that's every single day for two weeks every summer and every winter. We have partnerships with some of the biggest tech companies in Indonesia where we partner students with engineers and they work on projects together. And most recently, we're part of the MIT Solve program with our new initiative, Kelas. So Kelas, we're creating a free-to-use site for teachers. So right now, we have partnered with a handful of universities in Indonesia, both in rural and city, of Jakarta, where teachers can now have the knowledge and material to explain newer concepts that maybe they're less familiar with. 'Cause startup world changes rapidly. How you develop changes rapidly. So this is one thing that we're most excited about, because every teacher impacts thousands of students a year, and being able to teach the teachers and give them the resources that they need is something that's really important.

    5. LRLenny Rachitsky1:00:11

       That's incredible. It's currently just in Southeast Asia, is that right?

    6. CWCrystal Widjaja1:00:15

       Only in Indonesia, because frankly, this is where, this is where everyone needs the most support. I mean, globally, STEM is not well-received or, uh, welcoming at all to women. I think it's gotten worse over the past few decades. Like, it's below 18% of college graduates are women in computer science. So we're really trying to reach the youngest generation, because that's when you are told or informed that computer science is for specific types of people.

    7. LRLenny Rachitsky1:00:46

       It's really sad to hear that it's heading in the wrong direction. What do you think is contributing to that?

    8. CWCrystal Widjaja1:00:52

       I think there is still a lot of this mental model of what a computer scientist is able to do, and how much support they're given. So, it's been shown in studies that, uh, the youngest generation, middle school, high school, you are more likely to be given introductory STEM classes as a male than as a female. So, women just aren't targeted for STEM at that younger age, and so when they enter the high school or college classes for computer science, they're way behind. And that does not feel good. No one likes to be, like, the worst in the class. And so it's mo- more likely that you'll drop out. We've seen studies at Carnegie Mellon that actually would create introductory computer science classes before the college class starts, and for the women who did join those classes, they've actually graduated at similar rates as their male counterparts. So, it's really setting them up for success.

    9. LRLenny Rachitsky1:01:52

       If folks want to help, you said that there's a PayPal page. Is there any other sort of action people can take?

    10. CWCrystal Widjaja1:01:58

        Yes. Enterprise software, we love to teach iOS development, licensed software. We have hundreds of students a year. So, let us know.

23. 1:02:07 – 1:03:09

    Where to reach Crystal

    1. CWCrystal Widjaja1:02:07

    2. LRLenny Rachitsky1:02:07

       Awesome. And they can reach you on GenerationGirl.com?

    3. CWCrystal Widjaja1:02:10

       GenerationG- Girl.org.

    4. LRLenny Rachitsky1:02:12

       Crystal, uh, thank you so much for being here. I've taken enough of your time. Two last quick questions. Where can folks find you online if they want to reach out, and then other than the Generation Girl chat we just had, is there any other way folks can be helpful to you?

    5. CWCrystal Widjaja1:02:25

       Yes. Please find me at CrystalWijaya.com. You can reach out to me, and my email is there. Listeners, please do instrumentation correctly.

    6. LRLenny Rachitsky1:02:34

       (laughs)

    7. CWCrystal Widjaja1:02:34

       Please don't track your KPIs. Please track your user journeys and experiences. We'll have much funner things to talk about if you do that.

    8. LRLenny Rachitsky1:02:43

       Amazing. PSA. Thank you so much, Crystal.

    9. CWCrystal Widjaja1:02:46

       Thanks, Lenny. This was a blast.

    10. LRLenny Rachitsky1:02:48

        Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review, as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at LenysPodcast.com. See you in the next episode.

Episode duration: 1:03:09