Volumetric medical images contain an enormous amount of visual information that can discourage the exhaustive use of local descriptors for image analysis, comparison and retrieval. Distinctive features and patterns that need to be analyzed for nding diseases are most often local or regional, often in only very small parts of the image. Separating the large amount of image data that might contain little important information is an important task as it could reduce the current information overload of physicians and make clinical work more ecient. In this paper a novel method for detecting key-regions is introduced as a way of extending the concept of keypoints often used in 2D image analysis. In this way also computation is reduced as important visual features are only extracted from the detected key regions. The region detection method is integrated into a platform{independent, web-based graphical interface for medical image visualization and retrieval in three dimensions. This web- based interface makes it easy to deploy on existing infrastructures in both small and large-scale clinical environments. By including the region detection method into the interface, manual annotation is reduced and time is saved, making it possible to integrate the presented interface and methods into clinical routine and work ows, analyzing image data at a large scale.

1. Institute of
Information Systems
Region-based volumetric medical image
retrieval
Antonio Foncubierta Rodríguez
Henning Müller
Adrien Depeursinge

2. The need for retrieval Institute of
Information Systems
• Millions of medical
images are produced
everyday worldwide
• Quickly increasing
• 30% of world storage
capacity
• Retrieval methods can
improve reuse for:
• Training
• Decision support

3. Describing medical images Institute of
Information Systems
• Images contain large
amounts of information
• CT Scan: 512x512x200
= ~50 Million voxels
• In medical images,
features occur in small
zones:
• Irrelevant information:
discarded
• Relevant information:
locally described

4. Local description Institute of
Information Systems
• Common local image
analysis options:
• Dense sampling
• Salient key points (2D):
• SIFT
• Superpixels
• No preferred method in
exists in 3D for now
• But 3D data needs local
analysis even more

5. Current Challenges Institute of
Information Systems
• Point-based techniques:
• How are points chosen?
• How many points are enough?
• How to integrate information from neighborhoods?
• Segmentation-based techniques:
• Application-specific: not reusable for other image
types or anatomical parts
• Local descriptors of large regions become global
descriptors

6. Multiscale Salient Region Detector Institute of
Information Systems
• Saliency-based:
• Detects where features will be useful
• No a priori decision of how many regions
• Reusable in all images where saliency occurs
• Region-based:
• Relevant neighborhood is immediately provided
• Multiscale:
• Large and small complementary regions are detected

7. Methods overview Institute of
Information Systems

8. Methods Institute of
Information Systems
1. Resampling
• Cubic voxels
• 1mm side
For each scale s:
2. Difference of
Gaussians is
computed

9. Methods Institute of
Information Systems
3. Find regional minima
• Fill hole algorithm on
the DoG image
• Substract the DoG
image to the hole filled
• Result: Map or regional
minima
4. Find regional maxima
• Grind-peak algorithm

10. Methods Institute of
Information Systems
5. Logical OR on maxima
and minima
6. Opening
• Ball structuring element
• Radius proportional (r)
to scale
7. Label connected
components

11. Parameters Institute of
Information Systems
• Scale progression s
By default s ranges from 2 to 16 in powers of 2
• Thresholding parameter k
Controls the minimum saliency
Larger k values produce fewer regions
• Radius parameter r
Controls the minimum size of detected regions
Larger r values produce smooth, large regions only,
removing small ones

12. Examples Institute of
Information Systems

13. Examples Institute of
Information Systems

14. Comparison with segmentation Institute of
Information Systems

15. Integration into a retrieval Institute of
Information Systems
application
• Descriptors integrated in the detector:
• Basic descriptors: statistical moments of gray level
values
• Wavelet descriptors: energy of the wavelet
coefficients in each region

16. Integration into a retrieval Institute of
Information Systems
application

17. Conclusions Institute of
Information Systems
• Medical image retrieval requires local analysis
• A region-of-interest detector coupled with a
descriptor can enable retrieval:
• Multi-scale regions
• No predefined number of regions
• No predefined shape
• Good results compared to manual segmentation of
ROIs
• Integration into web-based retrieval system for
better adoption in clinical practice

18. Institute of
Information Systems
Thanks for your attention!
More information at http://medgift.hevs.ch
Antonio Foncubierta-Rodríguez, Henning Müller and Adrien Depeursinge, Region-based
volumetric medical image retrieval, in: SPIE Medical Imaging: Advanced PACS-based
Imaging Informatics and Therapeutic Applications, Orlando, FL, USA, SPIE, 2013