High-dimensional neural representations: nature and interpretability.

A. A set of data in dimension N (here, N=25 pixels on a 2D grid) is composed of multiple items differing by their color, size and shape. The neural net learns the data and extracts M features, here M=3. Case of entangled representations: the features are activated by diverse data points with mixed color, size, shape, and are thus not easily interpretable. Case of disentangled representations: the features extracted by the neural network are aligned with the independent and defining characteristics of the data. Conditional generation of new data with prescribed color, shape or size is then easy.

B. The state of a neural network with N neurons is a point in the high-dimensional space of the neuron activities r_i; here, N=3. Due to the constraints induced by the interactions between the neurons, these states can be embedded in one or more manifolds of dimensions much smaller than N.