Pointwise Mutual Information
Text Representation & Classical NLP DS practice problem on Onlearn.
Difficulty: medium.
Topics: Understanding Compute Pointwise Mutual Information, Pointwise Mutual Information, Joint Probability Distribution, Marginal Frequency, Independence Assumption, Log-Likelihood Ratio, Natural Language Processing, Information Theory, Probability and Statistics, Corpus Linguistics, Data Mining, Distributional Semantics, Feature Engineering, Statistical Dependency Measures, Co-occurrence Analysis, Vector Space Models.
Implement a function to compute the Pointwise Mutual Information (PMI) given the joint occurrence count of two events, their individual counts, and the total number of samples. PMI measures how much the actual joint occurrence of events differs from what we would expect by chance.