I keep meeting AI startups that proudly tell me "we built on top of Snowflake" or "we run on Databricks," and then, two sentences later, explain why their inference is slow.

These are not contradictory statements. They are cause and effect.

The lakehouse is a triumph of engineering. Snowflake, Databricks, and BigQuery solved a hard problem: running enormous analytical queries over petabytes with elastic compute and near-zero operational burden. If your job is to aggregate two billion rows, train a model on a quarter of history, or power a BI dashboard, there is no better class of tool. I am not here to talk anyone out of their warehouse.

I am here to talk you out of putting it in the serving path.

Why analytical engines are slow where it counts

A lakehouse is built for the opposite of what serving needs, and the reasons are structural - not tuning problems you can configure away.

Columnar storage is the first. Warehouses store data by column because analytics reads a few columns across millions of rows: "average revenue by region." That layout is a gift for scans and a curse for the serving pattern, which is "give me this one row, all of its columns, right now." Reassembling a single record out of columnar storage is precisely the work the format was designed to avoid.

The second is the optimizer. Cost-based, distributed query planners are extraordinary at turning a gnarly join into an efficient plan across a cluster. But that planning carries a fixed cost per query. Amortized across a three-second scan of a billion rows, it is invisible. Sitting in front of a single-row fetch, that overhead is the latency. You are paying the setup cost of a freight train to move one envelope.

The third is distributed coordination. The same machinery that lets a warehouse fan a query across a cluster imposes a latency floor that a co-located row store does not have.

Put it together and a warehouse that answers a two-billion-row aggregation in three seconds can take 300 milliseconds or more to hand you one user's feature vector. That is a fine number for a dashboard. It is a fatal number inside a page load.

What the serving path really demands

The serving path has different physics because it lives inside a user-facing request budget. Concretely, it wants:

Sub-100ms P95 on a warm cache. Not average - P95, because the tail is what users feel.
Transactional consistency with row-level reads and writes.
Cache-friendly, index-backed point lookups - B-tree, hash, or HNSW for vectors.

The canonical example is the feature store. At inference time you have an entity key - a user ID, an account, a session - and you need that entity's feature vector back in single-digit to low-double-digit milliseconds, at high QPS, with no tail spikes. This is exactly why mature ML platforms split into an offline store (full history, columnar, for training) and an online store (latest values, row-oriented, for serving). The industry did not invent that split for fun. It invented it because one storage engine cannot be optimal for both access patterns.

The two-tier pattern that works

The architecture that holds up is unglamorous, and that is the point.

Serving tier: an OLTP database - Postgres with the right extensions - co-located with your application. pgvector for embeddings behind an HNSW index, partial indexes on hot predicates, JSONB where the shape is irregular. This is the tier that sits in the request path.

Analytical tier: the lakehouse, doing what it is superb at - batch training, BI, and computing large or expensive features over full history.

Between them, a sync: you compute in the warehouse and materialize the results - features, aggregates, scores - into the serving store, where they are read at low latency.

The pgvector receipt is worth stating precisely. For the serving hot path - top-k nearest neighbors for a single query, with metadata filters, under 100ms - a co-located Postgres-plus-pgvector instance beats issuing a vector similarity query against the warehouse. Not because the warehouse "cannot do vectors." Most of them can now. It beats it because the serving path punishes exactly the per-query planning and distributed coordination overhead that the warehouse cannot shed. Right tool, right path.

The "reverse ETL" rebrand

If the two-tier pattern sounds familiar, you may know it by its marketing name. Reverse ETL - the Census and Hightouch category - is "compute in the warehouse, sync the results back into operational systems." Which is the serving tier and the analytical tier, with a bus between them, sold as a product.

I have no quarrel with reverse ETL. It is a clean implementation of the right idea. I only want to name it plainly: it is the two-tier pattern. It is not an alternative to the pattern, and it is not a way to make the warehouse serve. It is a way to get warehouse-computed values into a store that can.

"But the vendors are adding serving"

This is the strongest counterpoint, and it is the most interesting part of the story - because the vendors are conceding the argument in two different ways.

Snowflake is making the serious, hard attempt at true unification. Hybrid Tables, part of Unistore, pair a row store with columnar storage in a single table type and route queries by shape, delivering double-digit-millisecond point lookups alongside analytics. That is a real capability and worth watching. It is also the ambitious path: making one engine truly serve two masters.

Databricks took the other road, and the choice is telling. Rather than make the lakehouse serve, it spent roughly a billion dollars acquiring Neon - serverless Postgres - and shipped Lakebase, a fully managed Postgres OLTP layer that reached general availability in early 2026. Its online feature stores are now backed by it. Read that sequence again: the leading lakehouse vendor answered the serving problem by bolting a Postgres row store onto the side of the lakehouse.

That is not a refutation of the two-tier pattern. It is a billion-dollar validation of it, packaged as a single SKU.

So the fair read is this. The gap is closing on the packaging - one bill, one console, one governance model - not on the physics. Under the hood you still have a row-oriented serving tier and a columnar analytical tier, because those remain two different jobs. The open question is procurement, not architecture: do you rent both tiers from one vendor, or assemble them yourself? That is a legitimate decision with real trade-offs. It is not the same thing as believing one engine will do both jobs well.

Where this leaves you

   [ SERVING TIER ]                 [ ANALYTICAL TIER ]
   Postgres + pgvector    <----->   Lakehouse
   sub-100ms P95                    Snowflake / Databricks / BigQuery
   row-level, indexed               batch training, BI, big features
   in the request path
            ^                                |
            |____ sync computed features ____|
                    (a.k.a. "reverse ETL")

Two tiers. One optimized for the request budget, one for the scan. A sync between them. Whether you buy that as one platform or wire it together yourself, the shape does not change.

The failure mode I keep watching startups walk into is collapsing the two into one tool to save a line item - and then discovering that a single engine tuned for both is optimal for neither.

If you are trying to make one tool do both jobs, you are paying for the worst of both.

#AIInfrastructure #DataEngineering #MLOps #Postgres #MachineLearning

The lakehouse is fine. The lakehouse is not your AI infrastructure.