How I AIHow a visually impaired engineer builds personal software with Claude Code + Wispr Flow
CHAPTERS
- 0:00 – 2:58
Joe’s vision loss and how AI changes what’s possible day-to-day
Joe shares how losing central vision before college reshaped his life—and how modern multimodal AI now fills gaps that used to require magnifiers or help from others. The conversation frames AI as a practical accessibility layer that makes everyday tasks faster and more independent.
- •Joe lost most central vision from Leber’s hereditary optic neuropathy
- •AI tools (Gemini/ChatGPT) now replace many formerly difficult workflows
- •Screen-sharing and multimodal assistants reduce reliance on magnification alone
- •AI is described as making the “world a whole lot easier” for daily life
- 2:58 – 4:51
From robotics to software engineering: finding a new creative outlet
Joe explains pivoting from mechanical/robotics interests to computer science when hands-on work became harder. He describes how AI is narrowing the productivity gap between sighted and visually impaired engineers, both professionally and personally.
- •Intro CS course sparked a new path and sense of creative agency
- •Software became more accessible than mechanical work as vision declined
- •AI is steadily closing the gap for visually impaired developers
- •Joe leads AI enablement at Babylist to help engineers build with AI across the SDLC
- 4:51 – 7:22
Personal software as an accessibility unlock (and why it’s now worth building)
Claire and Joe discuss the “personal software” moment: many accessibility needs aren’t economically attractive for companies to solve, but individuals can now build tailored solutions. Joe positions micro-tools as faster than Swiss-army AI browsers for specific, repeated actions.
- •Accessibility tooling isn’t always commercially viable, creating gaps
- •AI enables individuals to build customized solutions for themselves
- •Micro-tools can outperform all-in-one AI browsers for latency/UX
- •Chrome extensions become a lightweight platform for personal automation
- 7:22 – 10:05
Demo: Slack image-description extension with follow-up Q&A
Joe demos a Slack-in-Chrome shortcut that describes images on demand, then supports follow-up questions conversationally. It reduces the need to ask coworkers to explain visuals and avoids tedious zoom-based inspection.
- •Keyboard shortcut triggers image description on a focused Slack message
- •Tool provides gist-level description plus details (e.g., stroller features)
- •Follow-up questions (e.g., intended child age) are supported
- •Built quickly in a single Claude Code session (~25 minutes)
- 10:05 – 10:50
Extending the idea to Figma: describing UI nodes for faster dev work
Joe previews a variant that works inside Figma to describe selected nodes with prompts tuned for UI implementation needs. The goal is fewer clicks and faster comprehension of design intent and properties.
- •Figma version describes nodes with design-focused prompting
- •Highlights CTA colors and relevant UI attributes for a full-stack engineer
- •Avoids multiple UI steps to access design metadata
- •Emphasizes one-shortcut workflows over manual navigation
- 10:50 – 12:16
Demo: AI spell-check anywhere with a single shortcut (screen-reader friendly)
Joe shows a minimalist spell checker that works in any web input field and announces progress via screen reader. It’s intentionally constrained—fix typos only—to preserve author intent while improving speed and accessibility versus heavier tools.
- •Shortcut sends text to OpenAI and replaces it with corrected text
- •Screen reader announces “processing” and “complete” states
- •Prompt is deliberately strict: fix typos, don’t rewrite content
- •Motivation: fewer clicks, better accessibility than typical grammar tools
- 12:16 – 14:38
Keyboard shortcuts, latency, and ROI: why these micro-apps beat manual workflows
Claire and Joe unpack why speed and consistent shortcuts matter, especially for accessibility. Joe explains how AI collapses the payback period: tasks that save minutes daily can now be built in ~30 minutes, making personal automation a no-brainer.
- •Running Slack in Chrome enables extensions to augment a “desktop-like” workflow
- •Shortcut-driven UX reduces latency and cognitive load
- •Joe compares early workaround (ChatGPT mini window) vs in-context extension
- •ROI math: 3 minutes/day saved is now worth building when it takes 30 minutes
- 14:38 – 16:13
What they’ll build live: a Slack link summarizer to prevent “save for later” pileups
Joe proposes a new extension: summarize external links in the currently focused Slack message so he can decide immediately whether to read or skip. The target is a fast, under-10-second experience that reduces deferred reading debt.
- •Detect external links in a focused Slack message
- •Fetch article content in background/hidden tabs
- •Send content to OpenAI for 3–5 key takeaways
- •Return results quickly to support in-the-moment decisions
- 16:13 – 20:29
Drafting the PRD by voice inside VS Code (with Copilot dictation)
Joe dictates a lightweight PRD using VS Code’s audio input, generating a structured spec with goals, requirements, and open questions. He highlights a screen-reader-driven preference: accept changes quickly and review in-document rather than in diff views.
- •Uses VS Code audio (Copilot) to dictate the PRD
- •Generated PRD includes goals, functional/non-functional requirements, and open questions
- •Adds requirements: accessible modal, include title + link, 3–5 takeaways
- •Workflow tip: reading diffs is harder with screen readers; review in the document instead
- 20:29 – 23:13
Claude Code + a reusable “Chrome extension” Skill: scaffolding with repeatable patterns
Joe switches to Claude Code, explicitly invoking a custom Claude Skill built from patterns across prior extensions. He also notes maintaining shared configuration (like API keys) via symlinks to avoid repetitive setup across projects.
- •Uses a dedicated Claude Skill to standardize Chrome extension scaffolding
- •Edits prompts in an editor via Ctrl+G for better accessibility and control
- •Keeps OpenAI key in shared config; reuses it across extensions via symlink
- •Avoids exposing secrets by restricting file access and extracting keys safely when needed
- 23:13 – 34:57
Making agentic dev tools accessible: hooks, consistent menus, and markdown folding
Joe describes small but critical accessibility enhancements: audible alerts when Claude needs input, predictable numeric choices, and relying on folding headings to navigate plans. Claire generalizes this to product design: consistent patterns reduce cognitive friction for all users.
- •Custom “ding” hook signals when Claude Code needs user input
- •Numeric options (1/2/3) provide consistent, screen-reader-friendly navigation
- •Ctrl+G edits plans/prompts as files, enabling search and fast changes
- •Markdown folding helps non-visual scanning and structured review
- 34:57 – 36:59
Loading the extension in Chrome Developer Mode and the first end-to-end run
Joe walks through enabling Developer Mode, loading an unpacked extension, and refreshing Slack so the new code is active. The first test mostly works but returns JSON displayed incorrectly, prompting a quick iterative fix.
- •Chrome Developer Mode enables local “unpacked” extension loading
- •New extensions require a tab refresh to activate in Slack
- •First run shows output but with formatting/JSON rendering issues
- •They capture the failure and immediately iterate rather than starting over
- 36:59 – 42:16
Debugging fast: screenshot-to-Claude workflow, structured JSON, and successful retest
Joe uses a custom slash command to paste screenshots into Claude Code (solving Windows clipboard vs WSL friction). After adjusting the OpenAI response handling to return structured JSON, the extension renders correctly and remains screen-reader accessible via ARIA-compliant modals.
- •Custom “Paste Image” slash command bridges Windows clipboard and WSL terminal
- •Fix focuses on changing the OpenAI response to structured JSON rather than text
- •Chrome extension reload/update loop is quick in Developer Mode
- •Accessibility verified: modal focus management and ARIA roles prevent reading behind the modal
- 42:16 – 49:17
Lightning round: MCPs for bypassing UI, multimodal reading with kids, and prompt reset strategy
In closing Q&A, Joe discusses MCPs as a way to pull content into simpler formats (often markdown) and reduce tool-switching—especially when UIs are complex. He shares a personal highlight: using Gemini to “read” any children’s book via live camera, and his pragmatic prompting tip—reset context when it’s stuck.
- •MCPs reduce context switching and can improve accessibility by bypassing complex UIs
- •Some enterprise tools are surprisingly accessible (e.g., Google Docs), others harder (e.g., Notion)
- •Multimodal AI enables reading children’s books via Gemini live page-by-page
- •Prompting technique: when context is poisoned, start fresh and carry forward learnings
