Skip to content

By playing, you agree to uListen's Terms and Privacy Policy and YouTube's Terms of Service.

a16za16z

Unlocking Creativity with Prompt Engineering

The growth of AI is changing modern life—including the job market. In just one example, there is a highly creative role emerging alongside AI: the prompt engineer. In this episode we explore the emerging importance of prompting with Guy Parsons, the early learnings of how to do it effectively, and where this field might be going. Will the prompt engineer be more like the highly sought after DevOps engineer, or a proficiency like Excel that you find on every resume? Listen in to hear Guy’s take. Interested in the prompt competition? Email us at podpitches@a16z.com. Topics Covered: 0:00 - Introduction 01:41 - DALL-E 2 Prompt Book 05:33 - Parallel skills 06:58 - 80/20 prompting 10:22 - New ways of prompting 13:00 - Pulling the AI slot machine 18:13 - Comparing models 21:03 - Requested features 27:44 - Learning with AI 28:44 - Practical use cases 32:08 - A top 1% prompt engineer 37:28 - The most popular images Resources: * DALL-E 2 Prompt Book: https://dallery.gallery/the-dalle-2-prompt-book/ * Find Guy on Twitter: https://twitter.com/GuyP * Image combination: https://twitter.com/GuyP/status/1612880405207580672 * Guy’s amorphous prompt experiment: https://twitter.com/GuyP/status/1608475973300948993 * Guy’s space duck: https://twitter.com/GuyP/status/1601342688225525761 * Prompt base: https://promptbase.com/ * Lexica: https://lexica.art/ Stay Updated: * Find us on Twitter: https://twitter.com/a16z * Find us on LinkedIn: https://www.linkedin.com/company/a16z * Subscribe on your favorite podcast app: https://a16z.simplecast.com/ * Follow our host: https://twitter.com/stephsmithio Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. For more details please see a16z.com/disclosures.

Guy ParsonsguestSteph Smithhost
Mar 9, 202339mWatch on YouTube ↗

Summaries, chapters, transcripts, and FAQs on this page are made by uListen, not YouTube.

CHAPTERS

  1. 0:00 – 4:12

    Why prompt engineers matter and how the Prompt Book began

    Steph Smith frames the prompt engineer as a creative role that emerges alongside AI. Guy Parsons explains how the DALL-E 2 Prompt Book grew from collecting examples and terms shared on social media into a slide deck of 80 to 100 slides.

    • •Steph Smith asks whether prompt engineering will be like DevOps or like Excel proficiency
    • •Text-to-image tools are basically a text box with no Photoshop-style controls
    • •Knowing art history and design language gives people a head start
    • •Guy collected examples from Twitter and Reddit into a slide deck he called a book
  2. 4:12 – 6:58

    Time spent prompting and parallel skills

    Steph asks how much time Guy has spent with the tools and what skill prompting resembles. Guy says he is no master, compares it to being good at googling, and describes it as finding the words that summon an image that already exists.

    • •Guy estimates a couple of hundred hours in the last six months
    • •Some Midjourney users have prompted thousands or hundreds of thousands of times
    • •Steph compares the situation to job listings demanding years of experience in a new language
    • •Guy likens prompting to being good at googling and to navigating an infinite Pinterest
  3. 6:58 – 10:23

    The 80/20 of prompting: describe it as if it already exists

    Guy gives his core tip: describe an image as if it already exists in a clip-art library or photo gallery with a caption. This explains why the tools struggle with step-by-step spatial instructions, and Steph adds points on alt-text training data and level of detail.

    • •Mimic captions that would accompany an existing image
    • •Real captions describe what an image is about, not how to draw it step by step
    • •Steph notes the models were trained on images paired with alt text
    • •Longer prompts show diminishing returns, according to Guy
  4. 10:23 – 12:49

    New ways of prompting: image prompts and selfie tools

    Guy describes how prompting has expanded to image inputs, which behave unpredictably but let users build on a visual base such as brand colors. He points to the selfie-avatar craze and startups that generate endless variations from a few core images.

    • •Image prompting does not collage images with words and is hard to control
    • •Brand colors or photos can serve as a base that is multiplied with other prompts
    • •Profile-picture tools prompt with about 20 selfies
    • •Startups generate infinite versions from about 10 core images
  5. 12:49 – 16:37

    Black-box randomness and pulling the AI slot machine

    Steph asks how to get more of the images you want from a black-box model. Guy explains that results start from random noise, so it is hard to tell skill from luck, and that seeing others' work helps. He also covers hard limits such as the hot dog example and negative prompts.

    • •The same prompt produces different pictures because each starts from different noise
    • •It is hard to know if someone else's great result came from cleverness or persistence
    • •Seeing others' prompts and work shows what is achievable
    • •Models can fail at simple things, like a hot dog drifting into dachshund features
    • •Negative prompts and glitches like hands are limits of the current tech
  6. 16:37 – 19:48

    Model quirks and comparing DALL-E, Midjourney and Stable Diffusion

    Guy explains that quirks depend on the model, such as DALL-E cutting off feet and heads in square frames, while Midjourney handles composition better. Asked whether switching tools is like Excel to Google Sheets or learning a new language, he says the principles are similar but the nuances differ, like driving different cars.

    • •DALL-E struggles with square framing; a white-border upload trick helps
    • •Midjourney iterated faster and solves composition problems such as group selfies
    • •Principles carry across tools, but specific effects differ due to training and fine-tuning
    • •Moving from DALL-E to Midjourney feels amazing, while the reverse feels worse
  7. 19:48 – 21:04

    Combining prompting with other tools

    Steph asks how text-to-image tools relate to the wider suite such as FaceTune, inpainting and outpainting. Guy sees an opportunity for Photoshop-like tools to work on the raw material generated, and notes that some looks are easier to get with other apps.

    • •The last 20% of refinement often needs another tool
    • •Photoshop-style tools presuppose an original image, which AI now supplies
    • •Vintage film looks are more easily achieved with existing iPhone apps
  8. 21:04 – 23:50

    Beyond the text box: requested features and unnamed styles

    Guy expects more models, with open-source Stable Diffusion enabling many new directions, and says the big challenge is going beyond the text box. He compares AI to clients who cannot explain what they want, and imagines conversational or multi-option interfaces that capture styles with no names.

    • •Open-source Stable Diffusion will lead to many specialized models
    • •Describing things with words is still hard, like clients who can't explain what they dislike
    • •Conversational interfaces and fast multiple options could help
    • •Some aesthetics have no name, and visual art expresses what words cannot
  9. 23:50 – 27:06

    Onboarding, style embeddings and locking in a look

    Steph proposes better onboarding and a tool for locking a style into a reusable variable, comparing it to Instagram filters sold by influencers. Guy describes the embedding trick behind the selfie craze, its possible use for style, and startups working on consistent looks.

    • •Steph suggests guided onboarding that teaches how prompts interact
    • •Influencers sold filters built on tuned variables; mood-board packs might be similar
    • •Embedding shows the model a few photos so it can recreate a person
    • •The same approach for style raises legal questions
    • •Scenario and Leonardo aim at consistent looks
  10. 27:06 – 28:43

    Learning with the AI and psychedelic exploration

    Using chess bots as an analogy, Steph asks whether the tools have shown Guy things he had not considered. Guy describes two modes, steering versus letting the model lead, and how repeated variations lead to a psychedelic visual journey.

    • •Chess bots surfaced moves humans had never considered
    • •Users either wait to see what the model shows or visualize and reject outputs
    • •DALL-E's variations tool can lead to bizarre visual journeys
  11. 28:43 – 32:25

    Practical use cases and where AI-generated art fits

    Steph asks about practical applications. Guy says AI use may often go unadvertised, like green screen, and suggests the hobby of making images may be the entertainment. They discuss real-world uses and the many levels at which people engage with design.

    • •AI tools may be a narrow part of the creative process and not publicized
    • •Guy suggests the hobby of generating may itself be the entertainment
    • •Steph points to e-commerce and 3D printing as uses beyond a toy
    • •Visual design appears at many levels, from cushion patterns to labels
    • •Sometimes the human touch is the point; other times a soothing pattern is enough
  12. 32:25 – 37:27

    Will prompt engineering be a top 1% skill or a universal one?

    Steph poses the question of whether prompting will be a rare, valuable skill or a universal one. Guy says model makers have incentive to make prompting unnecessary, but dedicated explorers will still push boundaries, and hidden prompt layers behind interfaces will be someone's job. They also speculate on niches and the 10X metaphor.

    • •Midjourney v4 produces impressive results from very simple prompts
    • •Guy doesn't think prompting will be a necessary skill for everyone
    • •Passionate explorers will go furthest, like expert specialists in other crafts
    • •Hidden prompt layers behind consumer interfaces may be a job
    • •Steph expects a bimodal outcome and niche specializations; Guy questions the 10X metaphor
  13. 37:27 – 39:41

    Memes, the most popular imagery, and wrap-up

    Steph asks about the most popular imagery online and suggests memes, showing that resonance does not require refined imagery. Guy jokes about 10X memes, and they close with links to the Prompt Book and Guy's social accounts.

    • •Guy guesses photos of parties; Steph suggests memes
    • •Memes are basic images with text, yet hugely popular
    • •Art is subjective and polish does not guarantee resonance
    • •Guy can be found on Twitter and on a Substack

Get more out of YouTube videos.

Read along on your phone, search it beside the video, or ask your favorite AI app what was said.