The 3D Slicer software (http://slicer.org) is a free package supporting research in a range of clinical domains ranging from neuroscience to image guided cancer therapy. In this talk I will describe the community of developers and users who have built and adapted 3D Slicer to enable their research. A functioning zSpace module in 3D Slicer now allows these scientist to explore the zSpace device. I’ll review target application domains such as image guided needle procedures for prostate and gynecological therapy and describe how people can use 3D Slicer and zSpace in their own research.

1. NA-MIC
National Alliance for Medical Image Computing
http://na-mic.org
3D Slicer and zSpace:
Medical Visualization and Clinical Applications
Steve Pieper, PhD
3D Slicer Chief Architect
Jayender Jagadeesan, PhD
Harvard Medical School
Isomics is a small technology development company working closely with academic and commercial institutions in
medical imaging research and software development.
Monday, October 21, 13

2. Goal and Outline
• Introduction to Our
Software and
Research (Steve)
• Example Research
in Image Guided
Therapy (Jay)
• Integration of
zSpace (Steve)
Images: A. Golby, F. Talos, P. Black
© 2013, All Rights Reserved
Monday, October 21, 13

3. Domestic Network of Peers
•
Leadership:
–
BWH: Ron Kikinis, (Overall PI)
•
Algorithms
–
Utah: Ross Whitaker (Core 1 PI)
–
–
–
–
•
MIT: Eric Grimson
UNC: Guido Gerig
MGH: Bruce Fischl, Dave Kennedy
GaTech: Allen Tannenbaum
•
•
•
•
Engineering
–
–
–
–
–
Kitware: Will Schroeder (Core 2 PI)
GE: Jim Miller
Isomics: Steve Pieper
UCSD: Mark Ellisman
UCLA: Art Toga
•
DBP 2004
–
BWH: Martha Shenton
–
Dartmouth: Andy Saykin
–
UCI: Steve Potkin
–
UofT: Jim Kennedy
•
•
DBP 2011
–
MGH: G. Sharp
–
Iowa: H. Johnson
–
Utah: R. MacLeod
–
UCLA: J. Van Horn
Service
–
Kitware: Stephen Aylward
Training
–
BWH: Sonia Pujol
Core 6 Dissemination
–
Isomics: Steve Pieper
–
BWH: Tina Kapur
Management
–
BWH: Sanjay Manandhar
DBP 2007
–
UNC: H. Cody
–
BWH: M. Kubicki
–
Mind Institute: J. Bockolt
–
Queens University: G. Fichtinger
© 2013, All Rights Reserved
Monday, October 21, 13
Provided by Pieper, Kikinis

4. International Network of
Peers
© 2013, All Rights Reserved
Monday, October 21, 13

5. What is 3D Slicer?
• A platform for image analysis and
visualization
• Current Release 4.3
• A freely-downloadable program
– Source code and executables available
for Windows, Linux, and Mac OS X
– All Code Reusable
• Slicer is a research platform:
– NOT an FDA approved medical device
– NOT finished – some parts will work
better than others
© 2013, All Rights Reserved
Monday, October 21, 13

6. What: FDA or Research
© 2013, All Rights Reserved
Monday, October 21, 13
Slide by Andras Lasso

7. What: FDA or Research
© 2013, All Rights Reserved
Monday, October 21, 13
Slide by Andras Lasso

8. Multi-Modality Visualization
•
Integrated Display of What is
Known About the Subject
– Anatomical Space as Common
Coordinate System
– Segmented Anatomy and
Volume Rendering for Context
– Statistics Volumes
– Interactive Visualization (View,
Visibility, Cropping, Slicing… )
– Therapy Plans, Measurements,
Annotations
– Analysis Results
– Linked Views
© 2013, All Rights Reserved
Monday, October 21, 13
Plesniak, Aucoin et al - BWH
Jakab and Berenyi - University of Debrecen

9. Registration
Preprocedure MR
•
Intraprocedure CT
Intra-subject
– Pre-Intra-Post Procedure
– Longitudinal Tracking of Disease Progression
•
Inter-subject
– Support Group Comparison (fMRI)
– Map Anatomical Atlas to Individual
•
Degrees of Freedom (DOFs)
– Rigid (Rotation + Translation)
– Similarity (Rigid + Uniform Scale)
– Affine (Rigid + Nonuniform Scale and Shear)
– Polyaffine (Locally Affine Interpolation)
– B-Spline (Cubic Displacement)
– Vector Field
© 2013, All Rights Reserved
Monday, October 21, 13
Sota - BWH

10. Navigation & Guidance
• OpenIGTLink Standard
– Tracked Instruments
– Real-Time Images
– Scanner and Robot
Control
• BWH AMIGO
– 3T MR + PET CT in OR
22
© 2013, All Rights Reserved
Monday, October 21, 13

11. Provided by Archip, Warfield
Intra-procedural Mapping
Alignment of all pre-operative
datasets to the intra-operative
images achieved during the
neurosurgery.
Archip N, Clatz O, Whalen S, Kacher
D, Fedorov A, Kot A, Chrisochoides N,
Jolesz F, Golby A, Black PM, Warfield
SK. Non-rigid alignment of preoperative MRI, fMRI, and DT-MRI with
intra-operative MRI for enhanced
visualization and navigation in imageguided neurosurgery. Neuroimage.
2007 Apr 1;35(2):609-24
© 2013, All Rights Reserved
Monday, October 21, 13

12. Extension Catalog
© 2013, All Rights Reserved
Monday, October 21, 13

13. 3D Slicer / AMIGO
Parathyroid
Adenoma Surgery
Cue Jay...
© 2013, All Rights Reserved
Monday, October 21, 13

14. 3D Slicer and
zSpace
© 2013, All Rights Reserved
Monday, October 21, 13

15. AMIGO CaseHub Display/
Control Options
Imaging
Modalities
MR, PET/CT, Fluroro,
US
PACS
Workstation for Technical Lead
Mouse/Keyboard/Monitor
SPL Compute
Resources
CaseHub Server
Wall Mounted Monitors
Display Only
Point of Care Interface
Touch/Touchless Interaction
Medical Records,
Real-Time Monitors,
Video
Vendor Workstations:
Navigation, Planning
(Brainlab, Siemens)
© 2013, All Rights Reserved
Monday, October 21, 13
• Use-Case Needs Assessment
• Clinician Control of Imaging
• Collaborative Data Management

16. AMIGO CaseHub Display/
Control Options
Imaging
Modalities
MR, PET/CT, Fluroro,
US
PACS
Workstation for Technical Lead
Mouse/Keyboard/Monitor
SPL Compute
Resources
CaseHub Server
Wall Mounted Monitors
Display Only
Point of Care Interface
Touch/Touchless Interaction
Medical Records,
Real-Time Monitors,
Video
Vendor Workstations:
Navigation, Planning
(Brainlab, Siemens)
© 2013, All Rights Reserved
Monday, October 21, 13
• Use-Case Needs Assessment
• Clinician Control of Imaging
• Collaborative Data Management
zSpace?

17. zSpace Slicer Prototype
Module
• Developed with Scott Dolim of zSpace
– Uses zSpace Python API
– Written as Slicer Scripted Module
– Integrates with Qt Event Loop
– Controls VTK Camera
• Not Yet Publicly Available
– zSpace to Create Slicer Extension
(Planned)
© 2013, All Rights Reserved
Monday, October 21, 13

18. zSpace - Slicer Status
• Already:
– Any Slicer Data in Displayable
– Easy to Use
– Ready for TLX-Style Human
Factors Testing
• Not Yet:
– No Stylus Support
– No GPU Volume Rendering
(Quad-Buffer Issue)
– Works with Python 2.6, not 2.7
© 2013, All Rights Reserved
Monday, October 21, 13

19. zSpace in AMIGO Issues
• No Table Space
– Would Need Boom Mounted Version
• Surgeon May Not Be Able To Move
– May Minimize Value of Head Tracking
• Glasses May Not Be Compatible
with Other Eyewear
• Polarized Glasses May Disrupt
Surgeon’s Normal Vision
• Only One Person Sees Correct
View
– Rest of Surgical Team Sees Garbage
© 2013, All Rights Reserved
Monday, October 21, 13

20. zSpace/Slicer Clinical
Research
• Utility Will Be Highly ProcedureDependent
• Screen Possible Target Areas
– Interventional
– Needle Placement
– Robot Control
– In-Scanner Procedures
© 2013, All Rights Reserved
Monday, October 21, 13

21. Clinical Research Strategy
• Phase I
– Bench Testing
– Pre-Procedure Planning/Review
• Phase II
– Place in AMIGO Control Room
– Intra-Procedural Imaging Review
• Phase III
– Navigation
– Patient Trials
© 2013, All Rights Reserved
Monday, October 21, 13

22. Summary
• 3D Slicer is Free and Open
Research Platform for Medical Image
Computing
• AMIGO Environment Ideal for Novel
Hardware/Software/Clinical Projects
• zSpace Integration Well Underway
and Promising
© 2013, All Rights Reserved
Monday, October 21, 13