Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (12)
Similaire à How we built a global search engine for genetic data
Similaire à How we built a global search engine for genetic data (20)
Plus de Miro Cupak
Plus de Miro Cupak (20)
Dernier
Dernier (20)
How we built a global search engine for genetic data
- 1. @mirocupak Miro Cupak VP Engineering, DNAstack 13/06/2018 How we built a global search engine for genetic data
- 2. @mirocupak What and why? !2 • Beacon Network • https://beacon-network.org/ • largest search and discovery engine of human genetic mutations • from the Global Alliance for Genomics & Health (GA4GH) • case study problem standard architecture technologies fun with stats
- 6. @mirocupak Trends !6 https://www.nature.com/news/technology- the-1-000-genome-1.14901 sequencing cost decreasing exponentially (3M times since 2000)
- 7. @mirocupak Trends !7 http://journals.plos.org/plosbiology/ article?id=10.1371/journal.pbio. 1002195 genomic data volume increasing exponentially (1M times since 2000)
- 8. @mirocupak Trends !8 DataVolumesby2025(GB) 0E+00 1E+10 2E+10 3E+10 4E+10 Twitter Youtube Genomics Lower Bound Upper Bound http://journals.plos.org/plosbiology/article? id=10.1371/journal.pbio.1002195 up to 2 billion human genomes sequenced in the next 10 years (more data annually than uploaded to and )
- 9. @mirocupak • no single institution will have sufficient resources • still, institutions don’t have enough data • common diseases • rare diseases • challenge • discovering data • solution • traditional approach of data aggregation in a single centralized site not working • federated system capable of executing cross-dataset and cross-institution queries is needed Problem !9
- 10. @mirocupak • nonprofit standards organization • a coalition of over 500 leading institutions working in health care, research, disease advocacy, life science, and information technology • goal: enable responsible sharing of genomic and clinical data • established in 2013 GA4GH & Beacon Project !10 • experiment to test the willingness of international sites to share genetic data in the simplest of all technical contexts • initiative requiring collaboration of many different GA4GH groups • started in 2014 and quickly gained traction http://ga4gh.org/ https://www.broadinstitute.org/files/news/pdfs/ GAWhitePaperJune3.pdf https://beacon-project.io/
- 12. @mirocupak • simple web service allowing users to query institution’s databases to determine whether they contain a genetic variant of interest • receives questions of the form Do you have information about this mutation? • responds with yes or no, optionally with additional information about the mutation • design principles • A beacon has to be technically simple. • A beacon has to minimize risks associated with genomic data sharing. • It has to be possible to make a beacon publicly available. Beacon !12
- 13. @mirocupak • no formal specification • receives questions of the form Do you have information about this mutation? • responds with yes or no • 4 public beacons, each API different Standard: Before Beacon Network !13 • request method • supported parameters • parameter names • chromosome identifiers • positional base • assembly notation • supported alleles • dataset support • response format • data included in the response
- 14. @mirocupak !14 Standard: Before Beacon Network
- 15. @mirocupak • 2014 • really simple (2 records) • true/false response • format: Avro • not enough traction • too vague • issues partially addressed by the Beacon Network Standard: 0.1 !15
- 16. @mirocupak • 2015 • true/false/overlap/null response • datasets • simple data use conditions • self description • format: Avro • complex (9 records) • not well adopted • not polished enough Standard: 0.2 !16
- 17. @mirocupak • 2016 • simplified 0.2 • based on real needs, successful • true/false/null response • data model improvements, extended metadata and response, improved support for datasets and cross-dataset queries, data versioning • modular and extensible • tooling • format: Avro → Proto3 Standard: 0.3 !17
- 18. @mirocupak • 2018 • stable and more flexible • support for more complex mutations • improved error handling • improved data use conditions • various minor improvements • developer experience • format: Proto3 → OpenAPI Standard: 0.4 !18
- 21. @mirocupak Data !21 • access data stored in a relational database
- 22. @mirocupak Service !22 • communication with other subsystems • query normalization • aggregators • participant resolution • query distribution • audit trail • L1 parallelization
- 23. @mirocupak Processor !23 • executing a query against a beacon and processing its response • management of a flexible, dynamic and easily extensible query execution pipeline • pipeline stages resolution (CDI and EJB) • L2 parallelization • cross-assembly query handling
- 24. @mirocupak Converter !24 • first stage in the query execution pipeline • translating query parameters
- 25. @mirocupak Requester !25 • second stage in the query execution pipeline • constructing beacon requests based on their URIs and parameters produced by the converters
- 26. @mirocupak Fetcher !26 • third stage in the query execution pipeline • unit actually talking to the API of beacons • submitting requests over the network and obtaining the raw response
- 27. @mirocupak Parser !27 • last stage in the pipeline • extracting information of interest from the raw response obtained by a fetcher • dealing with various formats • handling metadata, multiple responses, errors • response normalization • parallelized
- 28. @mirocupak Mapper !28 • translation between different representations of objects
- 29. @mirocupak REST !29 • handling client requests • data serialization
- 32. @mirocupak • 100 installations • 40 institutions • 18 countries • 6 continents Size !32
- 33. @mirocupak Users !33 • 13k users • 136 countries
- 36. @mirocupak Chromosomes !36 Others 39% Chr. 7 7% Chr. 13 11% Chr. 1 11% Chr. 17 14% Chr. 2 18%
- 37. @mirocupak Variants !37 Others 74% 2 : 212289100 C (ERBB4) 1% 2 : 29432776 C (ALK) 1% 14 : 23894969 A (MYH7) 1% 1 : 115258747 A (NRAS) 1% 1 : 43815163 C (MPL) 2% 7 : 140453136 C (BRAF) 2% 2 : 45895 G (FAM110C) 3% 22 : 46546565 A (PPARA) 3% 13 : 32936732 C (BRCA2) 6% 2 : 38938 C (FAM110C) 6% • 84k distinct mutations
- 38. @mirocupak Deleteriousness !38 Numberofvariants 1 1000 1000000 Score 0.00 0.07 0.14 0.21 0.28 0.35 0.42 0.49 0.56 0.63 0.70 0.77 0.84 0.91 0.98 Numberofvariants 1 1000 1000000 Score 0.00 0.07 0.14 0.21 0.28 0.35 0.42 0.49 0.56 0.63 0.70 0.77 0.84 0.91 0.98 SIFT (Sorting Intolerant From Tolerant) PolyPhen-2 HDIV (Polymorphism Phenotyping v2) 69% damaging, 31% tolerated 55% probably damaging, 22% possibly damaging, 23% benign
- 39. @mirocupak • 25% rare variants (1,000 Genomes Project) Rarity !39 Numberofvariants 1 100 10000 Allele frequency 0.00 0.03 0.06 0.090.12 0.15 0.18 0.21 0.240.27 0.30 0.33 0.36 0.39 0.420.45 0.48 0.51 0.54 0.57 0.600.63 0.66 0.69 0.72 0.75 0.780.81 0.84 0.87 0.90 0.93 0.960.99
- 40. @mirocupak Genes !40 Symbol Name 1 FAM110C Family With Sequence Similarity 110 Member C 2 BRCA1 BRCA1, DNA Repair Associated 3 BRCA2 BRCA2, DNA Repair Associated 4 PPARA Peroxisome Proliferator Activated Receptor Alpha 5 ERBB4 Erb-B2 Receptor Tyrosine Kinase 4 6 BRAF B-Raf Proto-Oncogene, Serine/Threonine Kinase 7 MPL MPL Proto-Oncogene, Thrombopoietin Receptor 8 MYH7 Myosin Heavy Chain 7 9 KIT KIT Proto-Oncogene Receptor Tyrosine Kinase 10 RET Ret Proto-Oncogene Others 53% RET 1% KIT 1% MYH7 2% MPL 2% BRAF 3% ERBB4 3% PPARA 4% BRCA2 9% BRCA1 10% FAM110C 11%
- 41. @mirocupak Disorders & clinical abnormalities !41 OMIM HPO 1 Pancreatic cancer, susceptibility to, 4 Autosomal dominant inheritance 2 Breast-ovarian cancer, familial, 1 Autosomal recessive inheritance 3 Fanconi anemia, complementation group D1 Scoliosis 4 Prostate cancer Short stature 5 Pancreatic cancer 2 Cognitive impairment 6 Medulloblastoma Constipation 7 Glioblastoma 3 Somatic mutation 8 Breast-ovarian cancer, familial, 2 Cafe-au-lait spot 9 Breast cancer, male, susceptibility to Failure to thrive 10 Wilms tumor Nausea and vomiting