Transformers Are Quietly Eating Recommendation Systems

When you open Netflix, Amazon, or Spotify and the next thing it suggests feels suspiciously right, what's running under the hood isn't what most people imagine. For a decade, recommendation systems lived on matrix factorization and collaborative filtering. Quietly, that has changed. The same transformer architecture that revolutionized language models has taken over sequential recommendation, and the benchmark gap is no longer subtle.

I want to walk through how we got here, why it matters, and where things are heading.

The classical era: matrix factorization

Until roughly 2016, the canonical approach to "users who liked X also liked Y" was matrix factorization. You take a giant sparse user-item matrix (users on one axis, items on the other, ratings as cells), and you decompose it into two smaller dense matrices, one row per user, one row per item, such that their dot product approximates the original ratings.

This was beautiful, fast, and explainable. It also had a hard ceiling: it doesn't know that you watched Inception yesterday and Tenet this morning. It treats your history as a bag of items, not a sequence.

The sequence era: RNNs and CNNs

By 2016, papers like GRU4Rec started treating recommendation as a sequence-prediction problem. Your interaction history is a sequence; the next item is the prediction target. This is the framing that NLP had been using forever, and recsys borrowed it wholesale.

Caser (2018) tried convolutional networks over the recent interaction window. GRU4Rec used recurrent networks. Both moved the needle, but they shared a bottleneck: the hidden state has to carry everything forward, and long sessions degrade.

The transformer turn

Then came SASRec (2018) and BERT4Rec (2019). SASRec applied a unidirectional self-attention transformer to next-item prediction. BERT4Rec went bidirectional, masked item prediction over the user's whole sequence, just like masked language modeling.

The results were striking. On MovieLens-1M, the standard sequential recsys benchmark, every architecture jump shows up clearly in NDCG@10:

Two things to notice. First, the gap between matrix factorization (BPR-MF: 0.39) and a transformer (BERT4Rec: 0.55) is roughly 42% relative improvement, not marginal. Second, the transformer wins primarily because self-attention lets the model look at any past interaction as easily as the most recent one. A user binge-watching one director isn't dragged down by the random rom-com they watched in 2019.

Why this matters in production

The benchmark gap is one thing; the deployment story is another. Three reasons transformer-based recommenders moved out of papers and into real systems.

They handle cold-start better. Attention over content features (titles, embeddings, categorical signals) means a new item can get recommended on day one if it's semantically close to known interest patterns.

They unify retrieval and ranking. Classical recsys was a pipeline: candidate generation, then a separate scoring model. Transformer architectures can fold both steps into one model, simplifying the stack.

They cross-pollinate with LLMs. When your model is already a transformer, you can pass user history through a foundation model, condition on natural-language queries, or generate recommendations directly. Google's TIGER (2023) and the wave of generative recommendation papers exist because this pipeline is now natural.

Where this goes next

Sequence and content fusion. Pure sequential recsys is being replaced by models that ingest both interaction history and rich content features (image, text, audio embeddings) through the same attention stack.

Generative recommendation. Instead of scoring a fixed candidate set, the model autoregressively generates item IDs. TIGER (Rajput et al., NeurIPS 2023) tokenizes items into semantic IDs and lets a small transformer generate the next one.

LLM-conditioned recsys. Recommend me something like Past Lives but funnier becomes a real query, not a marketing fantasy. We're not all the way there yet, but the pipeline exists.

The practical takeaway

If you're building anything that needs to learn what users want from what they've done, the default architecture has flipped. Matrix factorization is still in production at scale (it's cheap and explainable) but for any new system where ranking quality matters, the starting point should be a transformer trained on sequential interactions. Five years ago that was a research choice. Today it's the boring choice.

This is exactly the shift we've been working through at BrightLearn. When you're predicting what content a student should see next, that's a sequential recommendation problem at heart. The same architectures that beat collaborative filtering on movies are now beating handcrafted lesson sequencing in personalized learning.

References: Sun et al., BERT4Rec (CIKM 2019). Kang and McAuley, SASRec (ICDM 2018). Rajput et al., TIGER (NeurIPS 2023).