Combined Token Sampling Pipeline (Temperature + Top-k + Top-p)
Text Generation & NLP Evaluation DS practice problem on Onlearn.
Difficulty: hard.
Topics: Understanding Combined Probability Distribution Sampling for LLMs, Temperature Scaling, Top-k Truncation, Cumulative Distribution Function, Probability Re-normalization, Argmax vs Random Sampling, Probability Theory, Information Theory, Natural Language Processing, Statistical Inference, Optimization, Softmax Activation, Logit Scaling, Nucleus Sampling, Truncated Distribution, Stochastic Sampling.
Implement a function sample tokens(logits, k, p, temperature) that takes a list of raw logits and returns a single sampled index. The pipeline must follow these steps: 1. Apply temperature scaling to logits (dividing by T). 2. Convert to probabilities using Softmax. 3. Apply Top k filtering (keep only the k largest values). 4. Apply Top p (Nucleus) filtering (keep the smallest set of tokens whose cumulative probability exceeds p). 5. Re normalize the resulting distribution and sample a token using numpy.random.choice.