Deformable image registration (DIR) has many exciting potential applications in diagnostic medical imaging and radiation oncology. Automated propagation of physician-drawn contours to multiple image volumes, functional imaging, and 4D dose accumulation in thoracic radiotherapy are just a few examples. However, before such applications can be successfully and safely implemented, we require that the DIR spatial accuracy performance be rigorously and objectively assessed.

Objective evaluation of DIR is an active area of research. A framework for DIR evaluation is an essential utility for algorithm optimization, performing comparisons between algorithms, models, and implementations, acceptance testing prior to clinical implementation, and quality assurance of DIR on a routine basis.

Previously we reported a framework for DIR evaluation based on manual identification of large sets of prominent image features between volumetric image pairs1. The study demonstrates that considerable misrepresentation of DIR spatial accuracy performance characteristics can result from analyses based on inadequate landmark sample size and distribution.

Additionally, we have shown that large samples facilitate thorough characterization of spatial accuracy performance in terms of clinically relevant parameters, such as relative spatial location or displacement magnitude. Analyses based on landmark samples that are not sufficient in size, or that are biased towards structures that are generally easier to identify risk misrepresentation of the actual spatial accuracy performance of an algorithm.

These considerations make objective comparison of published DIR spatial accuracies difficult to interpret and potentially misleading. Therefore, we have established this website to provide a comprehensive common data set to investigators in this field who would like to evaluate their own algorithms, models, implementations, etc., using previously reported and characterized reference data sets composed of large samples of expert-identified landmark point pairs.

The database is a work in progress, and will be continually updated as more image volumes are manually annotated. It is crucial that reference data sets, such as ours, include as wide a range as possible of images encountered in clinical practice (i.e., image data with varying motion characteristics, image quality, disease states, clinical indications, etc.). Thus, we welcome any suggestions for particular data sets to make available on this site. Currently, we are working on additional datasets for multiple anatomic sites and imaging modalities. Registered users will be informed as new content is made available.

For more information, contact me directly at

1 Castillo R, Castillo E, Guerra R, Johnson VE, McPhail T, Garg AK, Guerrero T. 2009. A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets. Phys Med Biol 54 1849-1870.