← Back to Scratch Pad

Recommendation Systems

This blog posts does a pretty good job of laying out how a large-scale recommender system works: Real World Recommendation System - Part 1 - by Nikhil Garg (fennel.ai). In a nutshell, 

1. Generate candidates (narrow the search space) - can't run recommendation systems on all data; so filter the data using some heuristics. The filtered data is the 'inventory'. 
2. Filter the inventory - weed out stuff that's problematic, wrong, already seen by the user. 
3. Extract features - the data that's going to be fed to the recommendation models. Typically, once a recommendation system is launched, the filters are logged and persisted with the data when that data is created, updated. The production systems don't compute/generate features on the fly (kinda inefficient - and also introduces the risk of feature divergence).