At a glance
WHAT IT’S REALLY ABOUT
Shopify’s CTO on using big models to raise the ceiling
- Mikhail Parakhin explains Shopify’s AI philosophy as maximizing “ceiling-raising” breakthroughs—capabilities that were previously impossible—rather than only automating routine work.
- He describes how Shopify measures AI-driven productivity with internal delivery metrics that normalize output by project complexity, enabling more precise ROI discussions.
- Shopify’s policy of unlimited tokens and preference for the largest available models is designed to uncover ‘unknown unknowns’ and accelerate discovery during development.
- Parakhin highlights a flagship example: building a merchant ‘digital twin’ by modeling company actions as sequences, enabling counterfactual testing and targeted interventions that impact financial outcomes.
- He argues AI changes engineering management by elevating the need for technical acumen, clear intent, and a cost strategy that spends heavily in development while optimizing production inference.
IDEAS WORTH REMEMBERING
5 ideasAI’s highest ROI comes from “raising the ceiling,” not just automating drudge work.
Parakhin argues that the biggest impact isn’t making engineers faster at known tasks (floor-raising), but enabling work that was previously not feasible at any cost (ceiling-raising). He frames the best results as “centaur” workflows where the human and the model iterate together—neither succeeds alone.
Shopify measures AI productivity by projects shipped, normalized for complexity—not just PR volume.
Shopify tracks engineering throughput beyond raw PR counts by normalizing delivery against project complexity within their internal “Get Stuff Done” system. This helps quantify gains from AI-assisted execution even when the output changes shape (more projects, faster completion).
Use the largest model during development to discover what smaller models can’t do.
His recommendation is to default to the most capable model during exploration because the opportunity cost of missing “unknown unknowns” is high. Shopify operationalizes this with unlimited tokens for employees and heavy investment in inference/training infrastructure.
Sequence-modeling business actions enables a ‘digital twin’ that can recommend growth interventions.
Shopify models merchant/company behavior as a sequence of actions (analogous to text tokens) to build a “digital twin” that supports counterfactual interventions. They use it to test actions like loans, shipping improvements, or ad changes and then deploy the best interventions in production to drive merchant growth and business outcomes.
LLMs can materially improve engineering management by predicting slippage and surfacing risks early.
Parakhin says LLMs can surface weak signals (e.g., early indicators a project will slip) and prompt proactive intervention. He built a system to analyze what’s happening across projects and warn him before problems become visible through normal management channels.
WORDS WORTH SAVING
5 quotesIt's that you can do things that you previously couldn't, like, in no amount of time, no amount of helpers. Like, I could have had thousand, you know, best mathematicians at my disposal, I still wouldn't be able to do it. But with the model, I, I can.
— Mikhail Parakhin
It requires, you know, this back and forth and, uh, you know, creating that centaur, you know, using chess analogy, uh, that, that, that together, like, is better than the model and better than a human being.
— Mikhail Parakhin
Always use the largest model. Like, you... because the largest model raises the ceiling the most.
— Mikhail Parakhin
I don't think it was possible in the past... previously it would take infinity. We just wouldn't have done it. And then that's the def-- my definition of raising the ceiling, right? Like you start doing something that pre-previously was not even in consideration set.
— Mikhail Parakhin
My main advice would be try to not limit tokens.
— Mikhail Parakhin
High quality AI-generated summary created from speaker-labeled transcript.
