CHAPTERS
- 0:00 – 4:32
Why Jev week: fast, nearly free inference unlocks new product ideas
Claire and John set the stage for why Jev has dominated their attention: it’s extremely fast and cheap, making previously “not worth it” workflows suddenly viable. They contrast Jev’s economics with typical LLM usage and share early anecdotes about running massive data through the model.
- •Jev’s speed removes the “waiting around to call a function” pain
- •Costs are so low it changes what’s considered high-ROI experimentation
- •Large-scale data discovery becomes feasible (e.g., gigabytes of JSON)
- •Motivation: product-facing, real-time, and high-volume workflows
- 4:32 – 6:17
What Jev outputs: unstructured input → structured decisions (not text)
They clarify Jev’s core paradigm: it does not generate prose; it returns structured decisions, classifications, and confidence scores. This reframes the model as a bridge between human language and machine APIs/functions rather than a chatbot.
- •LLMs: unstructured→unstructured; Jev: unstructured→structured
- •Outputs are constrained options, probabilities, and scores
- •Best fit: mapping human language to known actions/functions
- •Key benefit: ‘talk to machines’ without building fragile prompts/chat UX
- 6:17 – 8:20
Demo: real-time voice-controlled to-do app (streaming classification)
John demos a dictation-driven to-do list that updates in real time without pausing or pressing enter. The app uses multiple layers of Jev inference to clean dictation, match tasks, select operations, and decide when enough information exists to execute an action.
- •Live dictation triggers actions mid-speech when confidence is sufficient
- •Multi-layer pipeline: validate transcript → match item → choose operation
- •Confidence scoring helps recover from dictation errors (e.g., item matching)
- •Demonstrates why Jev enables ‘instant’ tool calling UX
- 8:20 – 11:09
How sequential Jev calls chain together (think in if/else, build backwards)
Claire probes how the demo is architected, and John explains his mental model: Jev replaces many conditional branches and switch statements with probabilistic classification steps. He describes iterating from the end action backward and adding layers when needed to improve reliability.
- •Model as programmable classification: replace hand-written conditions
- •Build backwards from desired function calls and outcomes
- •Add more passes/layers instead of expecting one-shot perfection
- •Goal: practical assistants (e.g., better-than-Siri task management)
- 11:09 – 11:40
Demo: plain English → function name (grocery cart intent routing)
John shows a minimal example where user language like “What’s in my cart?” maps cleanly to a function such as getCart. This illustrates Jev as a foundational intent router that turns natural language into deterministic API calls.
- •Core primitive: map intents to known function names
- •Useful for command bars, assistants, and app control layers
- •Highlights Jev’s strengths with constrained action sets
- •Serves as a building block for larger agent/tool systems
- 11:40 – 13:45
Demo: data deduplication and record merging at scale
They explore using Jev for messy database cleanup—detecting near-duplicates and merging records (like contacts or companies). Claire underscores the value for pairwise comparisons across tens or hundreds of thousands of items that would be prohibitively expensive with typical LLMs.
- •Deduping/merging records via similarity classification across datasets
- •Scales to large volumes with millisecond-level per-decision performance
- •Strong fit for “messy data” workflows (contacts, passwords, company lists)
- •Enables clustering/grouping from pairwise comparisons
- 13:45 – 15:14
Confidence thresholds + multi-model validation for trust
They discuss using Jev’s confidence scores as a safety knob and layering additional checks. A common pattern: use Jev for broad/cheap matching, then validate samples or ambiguous cases with a stronger LLM and optionally generate explanations for auditability.
- •Confidence score drives merge threshold (e.g., only merge at 99%+)
- •Validate results with a more capable model after narrowing scope
- •Add qualitative explanations: ‘why these two match’
- •Iterative approach increases trust over repeated runs
- 15:14 – 18:09
Demo: Jev as a multi-level app router (tool selection → deep action)
John describes routing user input to the correct sub-tool or app module (e.g., ‘go to the to-do app’ and then execute an action). They generalize this to layered orchestration: Jev can select among tools, then select actions within tools, forming a fast navigation and command architecture.
- •Top-level routing: choose which tool/app module to invoke
- •Second-level routing: pick the specific action within the tool
- •Supports command-K/omnibar experiences and agent tool harnesses
- •Concept: ‘Jev is a router’ enabling deeper inferred navigation
- 18:09 – 19:48
Architecting around Jev: search, clustering, caching, and latency reality
They temper the hype: Jev is fast but not magically latency-free, so system design matters. Claire explains strategies like clustering and scoring groups rather than brute-force ranking every item, and both mention caching as essential for performance.
- •Real-time experiences require architecture, not just model calls
- •Search pattern: cluster/score groups, then refine (avoid exhaustive ranking)
- •Latency management via caching and precomputed structures
- •Advice: inspect GitHub demos to learn performance strategies
- 19:48 – 24:30
Demo: Jev vs traditional LLM at chess (speed/cost benchmarks, limited actions)
John pits Jev against a conventional LLM in blitz chess: Jev explores and ranks candidate moves quickly using a constrained move set and shallow lookahead. The comparison demonstrates why decision models shine when actions are enumerable and response time matters.
- •Two-step lookahead: rank top moves, explore next states, pick best
- •Jev completes moves sub-second; LLM is slower and more expensive
- •Benchmarks cited: ~10× faster average move, ~4× cheaper
- •Principle: constrained action spaces favor decision models over chat LLMs
- 24:30 – 28:34
DOM interactions as a decision set (browser use demystified)
They explain why ‘browser agents’ can work with decision models: while pixels look infinite, actionable DOM elements are limited. John contrasts this with tasks better suited to multimodal LLMs, like critiquing UI aesthetics from a screenshot.
- •Web navigation reduces to choosing among a small set of clickable elements
- •Games similarly map to controller inputs (constrained choices)
- •Heuristic: constrained inputs + app actions → reach for Jev
- •Use multimodal LLMs for open-ended visual critique and design feedback
- 28:34 – 28:56
Demo: Wikipedia “path to philosophy” route mapper (goal-directed crawling)
John demos a crawler that uses Wikipedia links to map a route from any topic to ‘Philosophy,’ using an end goal and iterative traversal. This highlights Jev’s ability to drive stepwise decision-making over APIs to reach a target state.
- •Goal-based routing: given a destination, choose next link/actions
- •Uses Wikipedia API to crawl and map routes
- •Illustrates ‘spidering’ through constrained options quickly
- •Shows applicability to navigation, planning, and graph traversal tasks
- 28:56 – 33:53
Demo: multi-agent coordination & collision avoidance (just-in-time control)
John shows a grid-based scenario where multiple agents must complete tasks without colliding. They discuss the value of cheap, repeated evaluation at each step—‘efficient inefficiency’—instead of relying on a single big-brain plan.
- •Assign tasks across multiple agents and prevent path collisions
- •Just-in-time re-evaluation each tick is viable due to low cost/latency
- •Connects to parallel agent swarms and coordination constraints
- •Tradeoff: local planning vs full-plan optimization; both become cheaper
- 33:53 – 37:25
Demo: real-time presentation coach + streaming windows and costs
John demonstrates a live speaking coach that checks off required bullet points as you talk, helping speakers stay on track under time pressure. They discuss implementation details for streaming: accumulating text until an action threshold, then sliding the window forward, enabled by ultra-low token costs.
- •Live transcription mapped to checklist completion in real time
- •Useful for interviews, workshops, talks—reduces rambling and misses
- •Implementation: concatenate words until actionable, then reset the window
- •Economics enable constant analysis (≈ pennies per million tokens)
- 37:25 – 46:06
Recap, where Jev falls short, and final lightning round (agents, hardware, mega.dev)
They recap key Jev patterns and discuss perceived shortcomings: single-pass setups may fail, so add layers and choose the right decision type (yes/no vs choice vs scoring). The conversation closes with a lightning round on agent paradigms, local hardware constraints, and John’s upcoming mega.dev program.
- •When Jev underperforms: add multi-pass classification instead of quitting
- •Try different decision forms (binary vs choice vs score) for accuracy
- •Broader trend: role-based multi-agent workflows (GroqBots, org design)
- •Practical reality: local compute/storage management for AI work
- •Closing: where to find John and mega.dev focus
