Skip to content
YC Root AccessYC Root Access

Diamond Maps: Efficient Reward Alignment for Generative Models

At our inaugural YCML at Startup School, YC Partner Ankit Gupta speaks with Douglas Chen about Diamond Maps, a method for steering generative models toward desired outputs more efficiently. Reward alignment depends on estimating how promising an intermediate state in the generation process is. Existing flow-map methods make this estimate using a single possible final output. Diamond Maps instead samples multiple outcomes from the same intermediate state, producing a better estimate and stronger guidance. The work includes both a fine-tuning method and a training-free inference-time method, allowing existing generative models to be aligned without necessarily retraining them. Apply to Y Combinator: https://www.ycombinator.com/apply Work at a startup: https://www.ycombinator.com/jobs

Ankit GuptahostDouglas Chenguest
Aug 6, 20265mWatch on YouTube ↗

At a glance

WHAT IT’S REALLY ABOUT

Diamond Maps enables efficient reward alignment via stochastic value estimation

  1. Diamond Maps targets reward alignment: steering a strong base generative model toward a user-defined task or style via reward guidance.
  2. Reward guidance depends on estimating a value function, but intermediate states in diffusion/flow-style generation are too noisy to score directly, so you typically must map to final samples first.
  3. Prior Flow Maps/flow matching approaches perform this mapping deterministically, yielding only one final sample per intermediate state and producing a weak value estimate.
  4. Diamond Maps makes the mapping stochastic so you can draw multiple final samples from the same intermediate state, improving value-function estimates and downstream alignment performance.
  5. The method is offered in two modes: a fine-tuning approach and a training-free inference-time plug-in for existing Flow Map models, trading training compute for inference compute.

IDEAS WORTH REMEMBERING

5 ideas

Better value estimates come from multiple futures, not one deterministic rollout.

Because an intermediate noisy state can lead to many plausible final images, sampling multiple final outcomes and averaging rewards yields a more faithful value function than a single deterministic completion.

Deterministic Flow Maps limit alignment by under-sampling uncertainty.

When only one final sample is produced per intermediate state, reward estimates become high-variance and can misguide alignment, reducing performance relative to multi-sample estimation.

Diamond Maps improves alignment measurably on standard metrics.

The talk cites improved LPIPS-based alignment compared to “vanilla” flow matching, suggesting the multi-sample value estimate translates into tangible steering gains.

Training-free alignment enables scaling when fine-tuning is too expensive.

Instead of re-training or fine-tuning large text-to-image models (compute-heavy), Diamond Maps can attach at inference to an existing Flow Map model, paying only extra inference-time compute.

Two deployment paths let teams choose where to spend compute budget.

You can either fine-tune from a pretrained Flow Map for efficiency at inference, or use the inference-only method for rapid iteration; doing both likely adds cost without much benefit per the discussion.

WORDS WORTH SAVING

5 quotes

Diamond Maps aims to solve this class of problems called reward alignment, and what reward alignment does is it essentially takes, uh, you know, a, a model, a base model, and it tries to steer or align it to a specific task that you might have.

— Douglas Chen

So for example, you might have a great, you know, image generation model, which generates great images, but you're not really able to steer it or prompt it to, you know, generate images that are interesting to you.

— Douglas Chen

Now, we have, you know, a lot of methods currently that help us estimate this value function, but the two, uh, main drawbacks are that they're either slow or they're also, uh, not great estimators with the value function because of... because they're deterministic.

— Douglas Chen

But Diamond Maps allows you to sample multiple final samples from this same, you know, intermediate-

— Douglas Chen

These prompts are chosen so that... you know, these are not images that will occur naturally. You know, you won't have a lighthouse in the middle of a desert, and you can see that it does extremely well on these, uh, examples.

— Douglas Chen

Reward alignment and reward guidanceValue function estimation challengesIntermediate noisy states vs final samplesFlow matching / Flow Maps baselineStochastic multi-sample mapping (Diamond Maps)Training vs training-free inference-time alignmentEmpirical results (LPIPS, prompt-aligned samples)

High quality AI-generated summary created from speaker-labeled transcript.

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.

iOSAndroidClaudeChrome