MHC Forward Pass
Vision-Language & Cross-Modal Systems DS practice problem on Onlearn.
Difficulty: medium.
Topics: MHC Forward Pass, Einsum Notation, Softmax Normalization, Gradient Accumulation, Contrastive Loss, KV Cache Quantization, Linear Algebra, Multimodal Deep Learning, Computational Complexity, Optimization Theory, Probability and Statistics, Tensor Operations, Cross-Attention Mechanisms, Backpropagation Algorithms, Feature Space Alignment, Memory Management.
Manifold Constrained Hyper Connections (mHC) extend residual connections by expanding to n parallel streams with learnable mixing. To ensure training stability, the mixing matrix H res is constrained to be doubly stochastic (rows and columns sum to 1) via Sinkhorn iteration. Implement the mHC forward pass: given n stream hidden states and raw coefficient matrices, apply the manifold projections and compute the output.