We propose a new methodology for automated landmark detection for breast MR images that combines statistical shape modelling and template matching into a single framework. The method trains a statistical shape model of breast skin surface using 30 manually labelled landmarks, followed by generation of template patches for each landmark. Template patches are matched across the unseen image to produce correlation maps. Correlation maps of the landmarks and the shape model are used to generate a first estimate of the landmarks referred to as ‘shape predicted landmarks’. These landmarks are refined using local maximum search in individual landmarks correlation maps. The algorithm was validated on 30 MR images using a leave-one-out approach. The results reveal that the method is robust and capable of localizing landmarks with an error of 3.41 mm ± 2.10 mm.

1. Automatic Landmark Detection
using Statistical Shape Modelling
and Template Matching
Authors
Habib Baluwala, Duane Malcolm, Jess Jor,
Poul Nielsen, Martyn Nash

2. Introduction
● Research Problem: Develop biomechanical models of
the human breast
● Construction of biomechanical model of the torso skin
surface
● Objective :
– Align the mean mesh of the skin surface with new image
data
– Match the mesh to the edges of the skin surface of the new
image

3. Challenges in Mesh Alignment
● Wide intensity range
● Variability of breast shapes
● Variability of torso shapes

4. Statistical Shape Modelling (SSM)
● Most structures of clinical interest have a
characteristic shape and anatomical location
relative to other structures
● Across the normal population the shape varies
statistically
● Provides prior knowledge of shape

5. Training Data
● Thirty 2D MRI training images
● The outline of the torso skin surface is represented by 24
manually labelled points
● We add another 6 anatomical landmarks:
– Sternum centre
– Aorta centre
– Spinal cord centre
– Vertebra centre
– Left and right nipple
● Total : 30 landmarks

6. Modelling Shape using PCA
● Compute the mean of the data
● Compute the covariance of the data
● Compute the eigenvectors and eigenvalues of the
covariance matrix, sorted in decreasing order of
eigenvalue size
● Remove the small eigenvalues, retaining most of the
variation
● is the mean shape, is a set of orthogonal modes of
variation and defines a set of components of
deformable model.

7. Torso Shape Model
+
+ +

8. Template Matching
● Move a template over an image and calculate
the similarity between the template and image
patch
● Similarity measures
– Cross Correlation (CC)
– Normalised Cross correlation (NCC)
– Sum of Squared Differences (SSD)
– Normalised Sum of Squared Differences (NSSD)

9. Average Template
+ + +
=
…..(30 images)

10. Template Matching
Vertebra centre
template
Test image
Template matching result
(correlation maps)

11. Template Matching
Aorta centre
template
Test image
Template matching result
(correlation maps)

12. Combining SSM and Template
matching
● Vary the mode weights for the first three shape
components
● Calculate the new shape and landmark
positions
● Move the correlation map to its respective
landmark location in the SSM shape model
● Multiply the correlation maps

13. SSM + Template matching
...
Shape Model
27 more landmark template matching results

14. SSM + Template Matching Results
(Shape Predicted Landmarks)
Manually selected landmarks and skin surface
Shape predicted landmarks

15. Local Maxima Search
● Crop the correlation map around the shape
predicted landmark (120 mm x 120 mm)
● Move the shape predicted landmark to the
local maximum of the correlation map
Shape predicted landmark Cropped correlation map Shape predicted
landmark + local maxima
search

16. SSM + Template Matching Results
+ Local Maxima Search
Move the shape predicted landmark to a local maximum
using correlation maps for individual landmarks
Manually selected landmarks and skin surface
Shape predicted landmarks + local maxima search

17. Results
Series of leave-one-out experiments performed on thirty
2D MRI images

18. Conclusion
● SSM + Template Matching + Local Maxima
search provides a robust detection of
landmark points on skin surface
● Average error = 3.4mm 2.1 mm
Future Work
● Extend the algorithm to 3D
● Incorporate active appearance models

19. Questions !!!