Fourier Basis Features for Value Function Approximation
Representation Learning, Advanced Theory & Miscellaneous DS practice problem on Onlearn.
Difficulty: medium.
Topics: Fourier Basis Features for Value Function Approximation, Fourier Basis Functions, Basis Order Selection, Curse of Dimensionality, Feature Scaling, Approximation Error Bounds, Reinforcement Learning, Functional Analysis, Numerical Optimization, Signal Processing, Statistical Learning Theory, Value Function Approximation, Orthogonal Basis Expansion, Spectral Analysis, Linear Function Approximation, Function Space Embedding.
Implement a function that computes the Fourier basis feature vector for a given state vector in a reinforcement learning setting. The Fourier basis provides a fixed set of basis functions for approximating value functions over a continuous state space. Given a d dimensional state vector s (where each component is normalized to [0, 1]) and an order parameter n, the Fourier basis consists of (n+1)^d features. Each feature is defined by a coefficient vector c (a d dimensional vector of non negative integers, each ranging from 0 to n). The feature value corresponding to coefficient vector c and state s is computed using a cosine function of their dot product scaled by pi. The coefficient vectors should be generated in standard lexicographic order (i.e., the rightmost index varies fastest, like counting in base n+1). Your function should: 1. Accept a 1D numpy array s of length d (state) and an integer order n 2. Generate all coefficient vectors in lexicographic order 3. Compute the corresponding feature values 4. Return the feature vector rounded to 4 decimal places as a 1D numpy array