A Tutorial on Conformal Prediction

Glenn Shafer and Vladimir Vovk 2008

The paper “A Tutorial on Conformal Prediction” by Glenn Shafer and Vladimir Vovk provides an in-depth overview of conformal prediction, a statistical technique that offers valid measures of uncertainty for predictive models. Conformal prediction uses past data to create prediction intervals or sets that contain the true value with a specified probability. This approach is non-parametric and makes minimal assumptions about the data distribution, primarily relying on the concept of exchangeability.

Key Concepts

• Nonconformity Measure: This measure assesses how different a new example is from the known examples. Various methods can be used to define this measure, such as distance to the nearest neighbor or residual error from a regression model.

• Conformal Algorithm: This algorithm generates prediction regions by evaluating the nonconformity scores of new examples against those of the training set. It ensures that the coverage probability of the prediction region matches the desired confidence level.

Conformal prediction can be applied to various machine learning methods, including nearest-neighbor classifiers, support vector machines, and ridge regression. It is particularly valuable in online learning scenarios where predictions are made sequentially. The tutorial provides practical examples and numerical demonstrations, illustrating how to implement conformal prediction in different scenarios. It emphasizes the importance of choosing an appropriate nonconformity measure and the impact of different choices on the prediction regions.

Conversation with Vladimir Vovk

2008 Paper

If you are unable to view the interview, please watch it here. If you are unable to view any of the documents above, please download the paper.