Triplet Loss

A contrastive learning method widely used in one-shot learning.

FaceNet, A Unified Embedding for Face Recognition and Clustering — Schroff, Kalenichenko, Philbin

This paper introduces the triplet loss function for contrastive learning, which, rather than using two data points as input, uses three: anchor, positive, and negative data points.

The introduced triplet loss function:

$${\mathcal{L}}_{triplet}(x,x^{+},x^{-})=\sum _{x\in \mathcal{X}}\text{max}(0,D(x_A,x_P)-D(x_A,x_N)+ϵ)$$

where again, D is the euclidian distance function aka L2 norm.

The goal of this loss function is to make sure the distance between the anchor and the negative is at least as big as the distance between the anchor and the positive, plus some safety measure $ϵ$. Euclidian distance is again used to pull together similar datapoints and pull apart dissimilar points. We get rid of indicator function and consider positive and negative points simultaneously.

Again, the $\text{max}(0,\cdot )$ is used to make sure we avoid negative values, as it does not make sense to optimize further once the condition is satisfied.

Crucially, you must select challenging enough negative data points to truly improve the model. This is known as the problem of hard negative mining.

References

FaceNet, A Unified Embedding for Face Recognition and Clustering