Unsupervised Learning

In supervised learning, we defined models that mapped observed data $x$ to output values $y$ and introduced loss functions that measured the quality of that mapping for a dataset $\{x_i,y_i\}$. Then, we discussed how to fit and measure the performance of these models.

The defining characteristic of unsupervised learning models is that they are learned from a set of observed data $\{x_i\}$ in the absence of labels. All unsupervised models share this property, but they have diverse goals. They may be used to generate plausible new samples from the dataset or to manipulate, denoise, interpolate between, or compress examples. They can also be used to reveal the internal structure of a dataset (e.g., by dividing it into coherent clusters) or to distinguish whether new examples belong to the same dataset or are outliers.

Latent variables

A common strategy in unsupervised learning is to define a mapping between the data examples $x$ and a set of unseen latent variables $z$. These latent variables capture the underlying structure in the dataset and usually have a lower dimension than the original data; in this sense, a latent variable $z$ can be considered a compressed version of a data example $x$ that captures its essential qualities.

Taxonomy of unsupervised learning models

In principle, the mapping between the observed and latent variables can be in either direction.

• Some models map from the data $x$ to the latent variables $z$. For example, the K-means algorithm maps the data $x$ to cluster assignment $z\in \{1,2,\dots ,K\}$.
• Other models map from the latent variables $z$ to the data $x$. Consider defining a distribution $\text{Pr}(z)$ over the latent variable $z$ in these models. New examples can now be generated by (i) drawing from this distribution and (ii) mapping the sample to the data space $x$. These are termed generative models.

dl
Latent variable
?

• Latent variables capture the underlying structure in the dataset
• Usually have a lower dimension than the original data; in this sense, a latent variable $z$ can be considered a compressed version of a data example $x$ that captures its essential qualities.
• Generative models map from latent variables $z$ to the data $x$.

+++