Skip to content
YC Root AccessYC Root Access

Zero-Shot Predictive Models for Relational Databases

At our inaugural YCML at Startup School, YC Partner Ankit Gupta speaks with Mark Žnidar about predictive models that work directly with relational databases without flattening their tables or manually engineering features. The method represents a relational database as a graph, samples the relevant neighborhood around an entity, and uses specialized attention mechanisms to capture columns, features, and relationships across tables. This preserves information that is often lost in traditional tabular machine learning pipelines. Despite having only 22 million parameters, the model outperformed a four-billion-parameter language model on the item-churn example shown in the presentation. Apply to Y Combinator: https://www.ycombinator.com/apply Work at a startup: https://www.ycombinator.com/jobs

Ankit GuptahostMark Žnidarguest
Aug 6, 20265mWatch on YouTube ↗

At a glance

WHAT IT’S REALLY ABOUT

Zero-shot transformer models predict directly from relational database structures accurately

  1. The speaker presents a model that uses relational databases (including foreign/primary key structure) directly rather than flattening tables for traditional feature engineering.
  2. Relational data is represented as a graph, and a neighborhood around the prediction target is sampled (e.g., via DFS/BFS) to gather relevant context from connected tables.
  3. The sampled graph context is serialized into a sequence of cells and embedded with type-specific encoders (numeric, text, Boolean) before being processed by a transformer.
  4. Novel masked attention mechanisms (column, feature, neighborhood attention) help the model learn distributions and selectively aggregate information from columns and neighboring entities.
  5. Results shown indicate the specialized small model can outperform general LLMs on relational prediction tasks (e.g., higher AUROC with ~22M parameters vs multi-billion-parameter LLMs).

IDEAS WORTH REMEMBERING

5 ideas

Keeping relational structure avoids information loss from flattening.

Traditional workflows often denormalize tables and hand-engineer features, which can discard important primary/foreign key relationships; treating the database as a graph preserves these signals.

Local neighborhood sampling provides task-relevant context efficiently.

Instead of ingesting the entire database, the model samples a context subgraph around the entity being predicted, capturing related records (e.g., a user and nearby connected entities) with controllable scope.

Serializing graph context enables transformer-based learning on databases.

After sampling, the system converts the neighborhood into a sequence of cells, making the input compatible with sequence models while still reflecting multi-table context.

Type-aware encoders are essential for mixed-schema data.

Relational databases contain heterogeneous value types; separate numeric, text, and Boolean encoders produce embeddings suited to each modality before attention-based aggregation.

Masked attention lets the model learn distributions and isolate signal sources.

Column attention can focus on a specific field across sampled neighbors to infer distributions, while feature and neighborhood attention separately emphasize the target entity’s attributes vs. connected entities’ information.

WORDS WORTH SAVING

5 quotes

So, um, we built the first, um, zero-shot predictive models for relational databases.

— Mark Žnidar

But we can say, "Hey, what about if we can make a model that doesn't require a data scientist, which kind of speeds up the exploration of your data, um, and maybe even gives you better predictions?"

— Mark Žnidar

So essentially, yeah, we develop a model which consumes the relational database as a primary modality.

— Mark Žnidar

It does very, very well. Like in comparison, for example, um, and this is also saying that LLMs are very bad- working with relational data.

— Mark Žnidar

So what is the kind of probability that, um, an item will churn is like the classification performance like 62 AUROC for Gemma, a four billion model, while it's at like 73 for our model.

— Mark Žnidar

Zero-shot prediction on enterprise databasesRelational DB as graph abstractionNeighborhood sampling (DFS/BFS)Serialization of table cells into sequencesType-specific encoders (number/text/Boolean)Masked attention: column/feature/neighborhoodBenchmarking vs LLMs (AUROC, parameter efficiency)

High quality AI-generated summary created from speaker-labeled transcript.

Get more out of YouTube videos.

High quality summaries for YouTube videos. Accurate transcripts to search & find moments. Powered by ChatGPT & Claude AI.

iOSAndroidClaudeChrome