BE - Natural Language Processing

Imagine you're building a text prediction system that predicts the next word after "I love my ___" using a trigram model, but your training data doesn't contain the exact phrase "I love my" (zero count problem). 

Backoff approach: The system first tries to use the trigram "I love my" to predict the next word, but since it's not found in training data, it backs off to use the bigram "love my" instead. If that also fails, it falls back to just looking at common words that follow "my" (unigram). So if "my cat" appears frequently in training data, it predicts "cat" based on the bigram "love my" → "cat". 

Interpolation approach: Instead of choosing one fallback, the system combines all levels: it takes 60% weight from the trigram "I love my" (even if zero, it gets a small smoothed value), 30% from the bigram "love my", and 10% from the unigram probability of common words. These weighted scores are added together to make the final prediction. 

Main difference: Backoff says "use trigram first, but switch to bigram if needed" while interpolation says "use trigram, bigram, and unigram all at once with different importance levels."