Entropy-based Split Selection

Tree Models & Ensembles DS practice problem on Onlearn.

Difficulty: medium.

Topics: Understanding Information Theory in Decision Trees, Logarithmic base-2 scaling, Relative frequency calculation, Weighted average of subsets, Handling zero-probability edge cases, Dataset partitioning logic, Information Theory, Decision Theory, Probability Distributions, Supervised Learning, Recursive Partitioning, Shannon Entropy, Information Gain, Gini Impurity, Node Purity, Class Probability Estimation.

Implement a function 'calculate entropy(labels)' that computes the Shannon entropy of a given list of class labels. Then, implement 'calculate information gain(parent labels, left labels, right labels)' to measure the effectiveness of a split.