Generated by Codex with GPT 5.6 Sol High

Meta’s official Engineering at Meta blog published “Meta Andromeda: Powering the Next Generation of Ads Retrieval Through Model-System Co-Design” on October 28, 2026. The post describes how Meta rebuilt the first stage of its advertising recommendation pipeline so that a single GPU server can evaluate more than one billion eligible ads per second, while using a model expressive enough to approach the quality of the much more expensive ranking stage.

The result did not come from tuning one layer in isolation. Meta changed the model architecture, fused model and non-model work into one GPU kernel, moved targeting and top-K selection onto the same machine, and optimized the resulting pipeline with NVIDIA against the limits of Grace Hopper hardware. Andromeda is therefore less a story about a faster neural network than about removing the boundaries between model, software system, and accelerator when those boundaries become the bottleneck.

Bringing ranking-like models into retrieval

An ads recommendation system cannot run its best ranking model against every possible ad. Retrieval first reduces a vast candidate pool to a much smaller set in milliseconds; ranking then spends more computation deciding which survivors should be shown. That division creates a quality ceiling: an ad discarded by retrieval never reaches the stronger model downstream.

Meta’s 2024 Andromeda system had already replaced conventional retrieval with a GPU-native design and expanded model capacity by four orders of magnitude. But the candidate pool continued to grow as generative tools made it easier to create ads. The new target was no longer tens of millions of evaluations per second per GPU, but billions, without giving up the richer interactions normally reserved for ranking.

The 2026 model uses a retrieval mixture-of-experts architecture. Instead of representing a person through one user tower, it builds an ensemble of specialized experts from signals such as sequential engagement history, cohort representations, semantic features derived by large language models, and activity across Meta surfaces. Each ad is encoded together with its current serving context. An attention-based gating network then uses the ad to activate the most relevant user experts and calculate the match score.

This sparse routing is the key tradeoff. The model can represent several distinct, changing aspects of a person’s interests without evaluating every expert for every candidate. Meta reports that it approaches ranking-stage matching quality with roughly 300 times fewer inference FLOPs than the 2024 retrieval model. The reduction is not merely a model-efficiency win; it creates the hardware headroom needed for much more aggressive systems optimization.

One interaction, one fused kernel

At billions of candidates per second, arithmetic was no longer the limiting resource. A conventional pipeline would launch separate kernels for embedding gathers, expert routing, interaction layers, and scoring. Each boundary materializes intermediate tensors in high-bandwidth memory. Even a small tensor multiplied by billions of ads produces terabytes per second of memory traffic.

Andromeda instead fuses the entire per-user-ad computation, including both neural-network operations and surrounding system logic, into one GPU kernel. Intermediate values remain in registers or thread-block shared memory rather than being written to and read from HBM. The kernel therefore spends the accelerator’s memory bandwidth on useful inputs instead of repeatedly moving temporary results.

The optimized gating kernel rose from 2.4 billion to more than 10 billion ad evaluations per second on one GH200, a 5.3-fold improvement, while reaching about 85% of peak HBM bandwidth. Meta also found that INT8 quantization could add as much as another twofold throughput gain without measurable accuracy loss relative to 16-bit precision. The striking part is that the more expressive gating network now runs about as fast as a simple dot-product scorer on the same hardware. Co-design has made its extra modeling complexity nearly free in the serving budget.

Collapsing the distributed pipeline

Kernel fusion exposed the next bottleneck. Per-request operations—including matching ads against advertiser targeting rules and selecting the top candidates—still ran on CPU hosts. Those hosts sent enormous candidate lists over the network to remote GPU scoring services. Once scoring reached multi-billion-candidate throughput, the network saturated before the GPU did.

Meta solved this by collapsing the retrieval system onto one GPU server. The CPU side sends only a compact request containing user features and a targeting query. The GPU performs index matching, fused model scoring, and top-K selection locally, then returns the small final candidate set. This reduces network traffic by about 100 times and makes GPU work, rather than data transport, the binding constraint again.

Moving index matching was not a mechanical port. Meta had to redesign a production index containing complex advertiser rules across tens of feature categories so that it supported massively parallel evaluation and real-time updates as ads were created, changed, or expired. After targeted kernel work, index matching exceeded 100 billion ads per second—an order of magnitude faster than the scoring network—leaving room for multiple retrieval pathways without turning targeting into the next choke point.

End-to-end profiling also showed that layout conversions and indirection between the matching and scoring stages consumed a meaningful share of latency. The teams co-designed data layouts across those components rather than optimizing each kernel behind a fixed interface. The production pipeline now scores every eligible ad at more than one billion ads per second on a single server; separately optimized components and the full benchmark pipeline retain additional headroom.

The engineering lesson is where to erase boundaries

Andromeda’s gains form a sequence. Sparse experts cut model FLOPs. Lower FLOPs make full kernel fusion practical. Fusion removes intermediate HBM traffic. Faster scoring exposes network transport as the next limit. Co-location removes that traffic. Shared layouts then eliminate overhead between the newly adjacent stages. Each improvement changes the system enough to reveal a different bottleneck.

That sequence is more useful than any individual technique. Hardware-aware engineering is often presented as making an existing model faster, but Meta’s design runs in both directions: the accelerator shapes the model, and the model’s sparse structure makes a different system architecture possible. The correct abstraction boundary at ordinary scale—between retrieval logic, model serving, and request processing—became an expensive data-movement boundary at Meta’s scale.

The broader takeaway is not that every recommendation system should put its entire pipeline in one GPU kernel. It is that mature systems should periodically re-test their architectural boundaries against the current bottleneck. When computation becomes cheap enough, moving data can dominate; when a network link becomes the ceiling, distributing work can be worse than co-locating it; and when hardware is underused, a more expressive model can sometimes occupy spare capacity without increasing serving cost. Andromeda’s 100-fold throughput gain over its 2024 predecessor came from treating those relationships as one design problem rather than a collection of local optimizations.