1. Genome in a Bottle
GIAB 2013
Workgroup 4: Performance Metrics

2. Q1.What performance metrics can/should
be generated when someone sequences
the GIAB RMs?

3. Sequence group 1: Initial characterization of the RM to
develop the ‘truth set’
EVERYTHING!

4. Chris Mason

5. Sequence group 2: People using reference materials to
benchmark tests.
Probably Not
Much

6. To do list:
Create a document describing metadata we want to capture (Chris Mason)
Identify fields we can reliably get from sequencers (Chris Mason)
Develop a flat data structure to capture information (Brad Chapman)
Help develop an improved individual genotype reporting format.
Work with CDC group on this.
Work with VCF/gVCF/GVF developers
✔

7. Q2. How should performance be
subdivided by region?
Q3. How should performance be
subdivided by variant type?

8. Assembly Region Reproducibility Track (for all RMs)
Highly confident regions
Less confident regions
Regions we can’t reliably call
NA12878 high quality genotype calls
Focus on SNVs and small indels first
Expand to other variant types as we get more confidence
Update definitions as we add additional reference materials.

9. Q4. How can GIAB help coordinate
the different groups developing
performance metrics?

10. Develop APIs for existing software:
X-prize/Harvard School of Public Health software
BCBio variation (comparison software)
O8 (visualization)
BCBio NextGen (Pipeline for running comparison)
Chris Mason’s software suite
Arvados software
GCAT software (Bioplanet)
GeT-RM browser for visualization