Top-p (Nucleus) Sampling
Text Generation & NLP Evaluation DS practice problem on Onlearn.
Difficulty: medium.
Topics: Understanding Top-p (Nucleus) Sampling for Text Generation, Nucleus Sampling, Temperature Scaling, Logit Bias, Argmax vs Sampling, Vocabulary Truncation, Cumulative Summation, Natural Language Processing, Probability Theory, Statistical Learning, Deep Learning Architectures, Optimization Algorithms, Language Modeling, Decoding Strategies, Stochastic Processes, Softmax Normalization, Cumulative Distribution Functions.
Implement a function top p sampling(logits, p) that performs nucleus sampling. Given a list of logits, convert them to probabilities using softmax, sort them in descending order, keep the smallest set of tokens whose cumulative probability exceeds or equals p, and return the indices and their re normalized probabilities.