AI / NLP Research
N-Gram Language Modeling
2024Case study
Stack
- Python
- NLTK
- Brown Corpus
- Probability Theory
- Perplexity
- Google Gemini API
Hands-on NLP research project that implements a trigram language model from scratch on the Brown corpus. It covers probability estimation, multiple smoothing strategies (Laplace, Kneser-Ney) and perplexity-based evaluation. A head-to-head comparison against modern LLMs through the Google Gemini API quantifies the gap between classical statistical models and contemporary transformers.
Next project
Information Retrieval System
View project→