SlideShare une entreprise Scribd logo
1  sur  57
Télécharger pour lire hors ligne
FluxGraph: A time-machine for your graphs
                   Davy Suvee
              Michel Van Speybroeck

                Janssen Pharmaceutica
about me

                 who am i ...
                 ➡ working as an it lead / software architect @ janssen pharmaceutica
                   • dealing with big scientific data sets
                   • hands-on expertise in big data and NoSQL technologies



                 ➡ founder of datablend
                   • provide big data and NoSQL consultancy
    Davy Suvee     • share practical knowledge and big data use cases via blog

      @DSUVEE
FluxGraph: a time-machine for your graphs
graphs and time ...
➡ graphs are continuously changing ...
graphs and time ...
       ➡ graphs are continuously changing ...

       ➡ graphs and time ...
          ★ neo-versioning by david montag  1
                                                                              2
          ★ representing time dependent graphs in neo4j by the isi foundation
          ★ modeling a multilevel index in neo4j by peter neubauer 3




1. http://github.com/dmontag/neo4j-versioning   2. http://github.com/ccattuto/neo4j-dynagraph/wiki   3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
graphs and time ...
       ➡ graphs are continuously changing ...

       ➡ graphs and time ...
          ★ neo-versioning by david montag  1
                                                                              2
          ★ representing time dependent graphs in neo4j by the isi foundation
          ★ modeling a multilevel index in neo4j by peter neubauer 3


                                                                            copy and relink semantics
                                                                                                     ๏ graph size
                                                                                                     ๏ object identity
                                                                                                     ๏ mixing data-model and time-model
1. http://github.com/dmontag/neo4j-versioning   2. http://github.com/ccattuto/neo4j-dynagraph/wiki      3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
FluxGraph ...
➡ towards a time-aware graph ...
FluxGraph ...
➡ towards a time-aware graph ...


➡ implement a blueprints-compatible graph on top of Datomic
FluxGraph ...
➡ towards a time-aware graph ...


➡ implement a blueprints-compatible graph on top of Datomic


➡ make FluxGraph fully time-aware
   ★ travel your graph through time
   ★ time-scoped iteration of vertices and edges
   ★ temporal graph comparison
travel through time
FluxGraph fg = new FluxGraph();
travel through time
FluxGraph fg = new FluxGraph();
                                   Davy

Vertex davy = fg.addVertex();
davy.setProperty(“name”,”Davy”);
travel through time
FluxGraph fg = new FluxGraph();
                                   Davy

Vertex davy = fg.addVertex();
davy.setProperty(“name”,”Davy”);
                                          Peter
Vertex peter = ...
travel through time
FluxGraph fg = new FluxGraph();
                                   Davy

Vertex davy = fg.addVertex();
davy.setProperty(“name”,”Davy”);
                                                    Peter
Vertex peter = ...
Vertex michael = ...

                                          Michael
travel through time
FluxGraph fg = new FluxGraph();
                                     Davy




                                                      kn
                                                       ow
Vertex davy = fg.addVertex();




                                                           s
davy.setProperty(“name”,”Davy”);
                                                       Peter
Vertex peter = ...
Vertex michael = ...

Edge e1 =                                   Michael
  fg.addEdge(davy, peter,“knows”);
travel through time

                                Davy
Date checkpoint = new Date();




                                                 kn
                                                  ow
                                                      s
                                                  Peter




                                       Michael
travel through time

                                    David
Date checkpoint = new Date();




                                                      kn
                                                       ow
                                                           s
davy.setProperty(“name”,”David”);                      Peter




                                            Michael
travel through time

                                       David
Date checkpoint = new Date();




                                                         kn
                                                          ow
                                                              s
davy.setProperty(“name”,”David”);                         Peter




                                       kn
Edge e2 =




                                        ow
  fg.addEdge(davy, michael,“knows”);




                                            s
                                               Michael
travel through time
time


                        kn
       Davy                  ow
                                  s



                                      Peter




              Michael
travel through time
time


                        kn
       Davy                  ow
                                  s




                                              checkpoint
                                      Peter




              Michael
travel through time
time


                        kn
       Davy                  ow                            David
                                                           Davy
                                  s




                                                                             kn
                                                                              ow
                                              checkpoint




                                                                                  s
                                      Peter                                   Peter




                                                           kn
                                                            ow
                                                                s
              Michael                                              Michael
travel through time
time


                        kn
       Davy                  ow                            David
                                                           Davy
                                  s




                                                                             kn
                                                                              ow
                                              checkpoint




                                                                                  s



                                                                                      current
                                      Peter                                   Peter




                                                           kn
                                                            ow
                                                                s
              Michael                                              Michael
travel through time                                           by default
time


                        kn
       Davy                  ow                            David
                                                           Davy
                                  s




                                                                             kn
                                                                              ow
                                              checkpoint




                                                                                  s



                                                                                          current
                                      Peter                                   Peter




                                                           kn
                                                            ow
                                                                s
              Michael                                              Michael
travel through time
time


                         kn
       Davy                   ow                            David
                                                            Davy
                                   s




                                                                              kn
                                                                               ow
                                               checkpoint




                                                                                   s



                                                                                       current
                                       Peter                                   Peter




                                                            kn
                                                             ow
                                                                 s
              Michael                                               Michael




                        fg.setCheckpointTime(checkpoint);
time-scoped iteration

       t1




     Davy
time-scoped iteration

       t1               t2


            change



     Davy            Davy’
time-scoped iteration

       t1               t2               t3


            change           change



     Davy            Davy’            Davy’’
time-scoped iteration

       t1               t2               t3                 tcurrrent


            change           change            change



     Davy            Davy’            Davy’’            Davy’’’
time-scoped iteration

         t1               t2               t3                 tcurrrent


              change           change            change



      Davy             Davy’            Davy’’            Davy’’’




  ➡ how to find the version of the vertex you are interested in?
time-scoped iteration
      t1        t2         t3        tcurrrent




    Davy     Davy’      Davy’’   Davy’’’
time-scoped iteration
      t1                 t2                 t3                   tcurrrent




             next              next                next

    Davy              Davy’              Davy’’              Davy’’’
           previous           previous            previous
time-scoped iteration
       t1                 t2                 t3                   tcurrrent




              next              next                next

     Davy              Davy’              Davy’’              Davy’’’
            previous           previous            previous




Vertex previousDavy = davy.getPreviousVersion();
time-scoped iteration
         t1                 t2                 t3                   tcurrrent




                next              next                next

       Davy              Davy’              Davy’’              Davy’’’
              previous           previous            previous




 Vertex previousDavy = davy.getPreviousVersion();
Iterable<Vertex> allDavy = davy.getNextVersions();
time-scoped iteration
            t1                 t2                 t3                   tcurrrent




                   next              next                next

          Davy              Davy’              Davy’’              Davy’’’
                 previous           previous            previous




     Vertex previousDavy = davy.getPreviousVersion();
   Iterable<Vertex> allDavy = davy.getNextVersions();
Iterable<Vertex> selDavy = davy.getPreviousVersions(filter);
time-scoped iteration
            t1                 t2                 t3                   tcurrrent




                   next              next                next

          Davy              Davy’              Davy’’              Davy’’’
                 previous           previous            previous




     Vertex previousDavy = davy.getPreviousVersion();
   Iterable<Vertex> allDavy = davy.getNextVersions();
Iterable<Vertex> selDavy = davy.getPreviousVersions(filter);
       Interval valid = davy.getTimerInterval();
time-scoped iteration
➡ When does an element change?
time-scoped iteration
➡ When does an element change?


➡ vertex:
   ★ setting or removing a property
   ★ add or remove it from an edge
   ★ being removed
time-scoped iteration
➡ When does an element change?


➡ vertex:                             ➡ edge:
   ★ setting or removing a property      ★ setting or removing a property
   ★ add or remove it from an edge       ★ being removed
   ★ being removed
time-scoped iteration
➡ When does an element change?


➡ vertex:                                ➡ edge:
   ★ setting or removing a property         ★ setting or removing a property
   ★ add or remove it from an edge          ★ being removed
   ★ being removed



➡ ... and each element is time-scoped!
temporal graph comparison

David
Davy                                          Davy




                                                                kn
                     kn




                                                                     ow
                      ow




                                                                      s
                          s
                      Peter   what changed?                          Peter
kn
 ow
     s




        Michael                                      Michael


           current                                      checkpoint
temporal graph comparison
➡ difference (A , B) = union (A , B) - B
➡ ... as a (immutable) graph!
temporal graph comparison
➡ difference (A , B) = union (A , B) - B
➡ ... as a (immutable) graph!
temporal graph comparison
➡ difference (A , B) = union (A , B) - B
➡ ... as a (immutable) graph!                   David




  difference (                  ,          )=




                                                kn
                                                 ow
                                                     s
FluxGraph ...

➡ available on github
      http://github.com/datablend/fluxgraph
use case: longitudinal patient data
    t1        t2        t3        t4        t5




          smoking   smoking             death




patient   patient   patient   patient   patient




                              cancer    cancer
use case: longitudinal patient data

➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)
use case: longitudinal patient data

➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)


➡ example analysis:
   ★ if a male patient is no longer smoking in 2005
   ★ what are the chances of getting lung cancer in 2010, comparing
        patients that smoked before 2005
        patients that never smoked
use case: longitudinal patient data
➡ get all male non-smokers in 2005

fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
use case: longitudinal patient data
➡ get all male non-smokers in 2005

fg.setCheckpointTime(new DateTime(2005,12,31).toDate());

Iterator<Vertex> males =
  fg.getVertices("gender", "male").iterator()
use case: longitudinal patient data
➡ get all male non-smokers in 2005

fg.setCheckpointTime(new DateTime(2005,12,31).toDate());

Iterator<Vertex> males =
  fg.getVertices("gender", "male").iterator()

while (males.hasNext()) {
   Vertex p2005 = males.next();
   boolean smoking2005 =
     p2005.getEdges(OUT,"smokingStatus").iterator().hasNext();
}
use case: longitudinal patient data
➡ which patients were smoking before 2005?


boolean smokingBefore2005 =
  ((FluxVertex)p2005).getPreviousVersions(new TimeAwareFilter() {

    public TimeAwareElement filter(TimeAwareVertex element) {
      return element.getEdges(OUT, "smokingStatus").iterator().hasNext()
        ? element : null;
    }

  }).iterator().hasNext();
use case: longitudinal patient data
➡ which patients have cancer in 2010

                                       working set of smokers
 Graph g =
   fg.difference(smokerws,
                 time2010.toDate(),
                 time2005.toDate());
use case: longitudinal patient data
➡ which patients have cancer in 2010

                                       working set of smokers
 Graph g =
   fg.difference(smokerws,
                 time2010.toDate(),
                 time2005.toDate());



➡ extract the patients that have an edge to the cancer node
FluxGraph: a time-machine for your graphs
gephi plugin for fluxgraph   2010
gephi plugin for fluxgraph   2001
gephi plugin for blueprints!

                                          1
     ➡ available on github
      http://github.com/datablend/gephi-blueprints-plugin

     ➡ Support for neo4j, orientdb, dex, rexter, ...

1. Kudos to Timmy Storms (@timmystorms)
Questions?

Contenu connexe

En vedette

Graph DB + Bioinformatics: Bio4j, recent applications and future directions
Graph DB + Bioinformatics:  Bio4j, recent applications and future directions Graph DB + Bioinformatics:  Bio4j, recent applications and future directions
Graph DB + Bioinformatics: Bio4j, recent applications and future directions Pablo Pareja Tobes
 
Graph databases in computational bioloby: case of neo4j and TitanDB
Graph databases in computational bioloby: case of neo4j and TitanDBGraph databases in computational bioloby: case of neo4j and TitanDB
Graph databases in computational bioloby: case of neo4j and TitanDBAndrei KUCHARAVY
 
PresentationFinal
PresentationFinalPresentationFinal
PresentationFinalLin Han
 
Cancer Research Data Ecosystem - Dr. Warren Kibbe
Cancer Research Data Ecosystem - Dr. Warren KibbeCancer Research Data Ecosystem - Dr. Warren Kibbe
Cancer Research Data Ecosystem - Dr. Warren Kibbeimgcommcall
 
Visual Exploration of Clinical and Genomic Data for Patient Stratification
Visual Exploration of Clinical and Genomic Data for Patient StratificationVisual Exploration of Clinical and Genomic Data for Patient Stratification
Visual Exploration of Clinical and Genomic Data for Patient StratificationNils Gehlenborg
 
Linked Cancer Genome Atlas Database
Linked Cancer Genome Atlas DatabaseLinked Cancer Genome Atlas Database
Linked Cancer Genome Atlas DatabaseMuhammad Saleem
 
LSQ: The Linked SPARQL Queries Dataset
LSQ: The Linked SPARQL Queries DatasetLSQ: The Linked SPARQL Queries Dataset
LSQ: The Linked SPARQL Queries DatasetMuhammad Saleem
 
Introduction to STINGER
Introduction to STINGERIntroduction to STINGER
Introduction to STINGERrobertmccoll
 
Clinical research training - Dr Blanaid Mee - Dec 7th 2016
Clinical research training - Dr Blanaid Mee - Dec 7th 2016Clinical research training - Dr Blanaid Mee - Dec 7th 2016
Clinical research training - Dr Blanaid Mee - Dec 7th 2016ipposi
 
City of hope research informatics common data elements
City of hope research informatics common data elementsCity of hope research informatics common data elements
City of hope research informatics common data elementsAbdul-Malik Shakir
 
Patient profiling disaggregating the data
Patient profiling disaggregating the dataPatient profiling disaggregating the data
Patient profiling disaggregating the datanhsnwHELP
 
Patient-Generated Data for Cancer Treatment and Management
Patient-Generated Data for Cancer Treatment and ManagementPatient-Generated Data for Cancer Treatment and Management
Patient-Generated Data for Cancer Treatment and ManagementTommy Snitz
 
Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...
Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...
Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...Emad Shash
 
iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transformi...
iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transformi...iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transformi...
iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transformi...Health IT Conference – iHT2
 
Elective Care Conference: the role of the MDT coordinator role
Elective Care Conference: the role of the MDT coordinator role Elective Care Conference: the role of the MDT coordinator role
Elective Care Conference: the role of the MDT coordinator role NHS Improvement
 
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas Weber
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas WeberGraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas Weber
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas WeberNeo4j
 
2015 Micromedex使用者大會 如何在臨床工作中找到實證解答
2015 Micromedex使用者大會 如何在臨床工作中找到實證解答2015 Micromedex使用者大會 如何在臨床工作中找到實證解答
2015 Micromedex使用者大會 如何在臨床工作中找到實證解答建豪 陳
 

En vedette (20)

Graph DB + Bioinformatics: Bio4j, recent applications and future directions
Graph DB + Bioinformatics:  Bio4j, recent applications and future directions Graph DB + Bioinformatics:  Bio4j, recent applications and future directions
Graph DB + Bioinformatics: Bio4j, recent applications and future directions
 
Graph databases in computational bioloby: case of neo4j and TitanDB
Graph databases in computational bioloby: case of neo4j and TitanDBGraph databases in computational bioloby: case of neo4j and TitanDB
Graph databases in computational bioloby: case of neo4j and TitanDB
 
Malmo 11.11.2008
Malmo 11.11.2008Malmo 11.11.2008
Malmo 11.11.2008
 
PresentationFinal
PresentationFinalPresentationFinal
PresentationFinal
 
Cancer Research Data Ecosystem - Dr. Warren Kibbe
Cancer Research Data Ecosystem - Dr. Warren KibbeCancer Research Data Ecosystem - Dr. Warren Kibbe
Cancer Research Data Ecosystem - Dr. Warren Kibbe
 
Data collection
Data collectionData collection
Data collection
 
Visual Exploration of Clinical and Genomic Data for Patient Stratification
Visual Exploration of Clinical and Genomic Data for Patient StratificationVisual Exploration of Clinical and Genomic Data for Patient Stratification
Visual Exploration of Clinical and Genomic Data for Patient Stratification
 
how to sell
how to sellhow to sell
how to sell
 
Linked Cancer Genome Atlas Database
Linked Cancer Genome Atlas DatabaseLinked Cancer Genome Atlas Database
Linked Cancer Genome Atlas Database
 
LSQ: The Linked SPARQL Queries Dataset
LSQ: The Linked SPARQL Queries DatasetLSQ: The Linked SPARQL Queries Dataset
LSQ: The Linked SPARQL Queries Dataset
 
Introduction to STINGER
Introduction to STINGERIntroduction to STINGER
Introduction to STINGER
 
Clinical research training - Dr Blanaid Mee - Dec 7th 2016
Clinical research training - Dr Blanaid Mee - Dec 7th 2016Clinical research training - Dr Blanaid Mee - Dec 7th 2016
Clinical research training - Dr Blanaid Mee - Dec 7th 2016
 
City of hope research informatics common data elements
City of hope research informatics common data elementsCity of hope research informatics common data elements
City of hope research informatics common data elements
 
Patient profiling disaggregating the data
Patient profiling disaggregating the dataPatient profiling disaggregating the data
Patient profiling disaggregating the data
 
Patient-Generated Data for Cancer Treatment and Management
Patient-Generated Data for Cancer Treatment and ManagementPatient-Generated Data for Cancer Treatment and Management
Patient-Generated Data for Cancer Treatment and Management
 
Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...
Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...
Impact of Multidisciplinary Discussion on Treatment Outcome For Gynecologic C...
 
iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transformi...
iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transformi...iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transformi...
iHT² Health IT Summit New York - Cancer Care Ontario Presentation "Transformi...
 
Elective Care Conference: the role of the MDT coordinator role
Elective Care Conference: the role of the MDT coordinator role Elective Care Conference: the role of the MDT coordinator role
Elective Care Conference: the role of the MDT coordinator role
 
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas Weber
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas WeberGraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas Weber
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas Weber
 
2015 Micromedex使用者大會 如何在臨床工作中找到實證解答
2015 Micromedex使用者大會 如何在臨床工作中找到實證解答2015 Micromedex使用者大會 如何在臨床工作中找到實證解答
2015 Micromedex使用者大會 如何在臨床工作中找到實證解答
 

Dernier

Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfinfogdgmi
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Websitedgelyza
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxUdaiappa Ramachandran
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfDianaGray10
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-pyJamie (Taka) Wang
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding TeamAdam Moalla
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 

Dernier (20)

Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Videogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdfVideogame localization & technology_ how to enhance the power of translation.pdf
Videogame localization & technology_ how to enhance the power of translation.pdf
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
COMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a WebsiteCOMPUTER 10 Lesson 8 - Building a Website
COMPUTER 10 Lesson 8 - Building a Website
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Building AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptxBuilding AI-Driven Apps Using Semantic Kernel.pptx
Building AI-Driven Apps Using Semantic Kernel.pptx
 
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdfUiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
UiPath Solutions Management Preview - Northern CA Chapter - March 22.pdf
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
20230104 - machine vision
20230104 - machine vision20230104 - machine vision
20230104 - machine vision
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
20230202 - Introduction to tis-py
20230202 - Introduction to tis-py20230202 - Introduction to tis-py
20230202 - Introduction to tis-py
 
9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team9 Steps For Building Winning Founding Team
9 Steps For Building Winning Founding Team
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 

FluxGraph: a time-machine for your graphs

  • 1. FluxGraph: A time-machine for your graphs Davy Suvee Michel Van Speybroeck Janssen Pharmaceutica
  • 2. about me who am i ... ➡ working as an it lead / software architect @ janssen pharmaceutica • dealing with big scientific data sets • hands-on expertise in big data and NoSQL technologies ➡ founder of datablend • provide big data and NoSQL consultancy Davy Suvee • share practical knowledge and big data use cases via blog @DSUVEE
  • 4. graphs and time ... ➡ graphs are continuously changing ...
  • 5. graphs and time ... ➡ graphs are continuously changing ... ➡ graphs and time ... ★ neo-versioning by david montag 1 2 ★ representing time dependent graphs in neo4j by the isi foundation ★ modeling a multilevel index in neo4j by peter neubauer 3 1. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
  • 6. graphs and time ... ➡ graphs are continuously changing ... ➡ graphs and time ... ★ neo-versioning by david montag 1 2 ★ representing time dependent graphs in neo4j by the isi foundation ★ modeling a multilevel index in neo4j by peter neubauer 3 copy and relink semantics ๏ graph size ๏ object identity ๏ mixing data-model and time-model 1. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
  • 7. FluxGraph ... ➡ towards a time-aware graph ...
  • 8. FluxGraph ... ➡ towards a time-aware graph ... ➡ implement a blueprints-compatible graph on top of Datomic
  • 9. FluxGraph ... ➡ towards a time-aware graph ... ➡ implement a blueprints-compatible graph on top of Datomic ➡ make FluxGraph fully time-aware ★ travel your graph through time ★ time-scoped iteration of vertices and edges ★ temporal graph comparison
  • 10. travel through time FluxGraph fg = new FluxGraph();
  • 11. travel through time FluxGraph fg = new FluxGraph(); Davy Vertex davy = fg.addVertex(); davy.setProperty(“name”,”Davy”);
  • 12. travel through time FluxGraph fg = new FluxGraph(); Davy Vertex davy = fg.addVertex(); davy.setProperty(“name”,”Davy”); Peter Vertex peter = ...
  • 13. travel through time FluxGraph fg = new FluxGraph(); Davy Vertex davy = fg.addVertex(); davy.setProperty(“name”,”Davy”); Peter Vertex peter = ... Vertex michael = ... Michael
  • 14. travel through time FluxGraph fg = new FluxGraph(); Davy kn ow Vertex davy = fg.addVertex(); s davy.setProperty(“name”,”Davy”); Peter Vertex peter = ... Vertex michael = ... Edge e1 = Michael fg.addEdge(davy, peter,“knows”);
  • 15. travel through time Davy Date checkpoint = new Date(); kn ow s Peter Michael
  • 16. travel through time David Date checkpoint = new Date(); kn ow s davy.setProperty(“name”,”David”); Peter Michael
  • 17. travel through time David Date checkpoint = new Date(); kn ow s davy.setProperty(“name”,”David”); Peter kn Edge e2 = ow fg.addEdge(davy, michael,“knows”); s Michael
  • 18. travel through time time kn Davy ow s Peter Michael
  • 19. travel through time time kn Davy ow s checkpoint Peter Michael
  • 20. travel through time time kn Davy ow David Davy s kn ow checkpoint s Peter Peter kn ow s Michael Michael
  • 21. travel through time time kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael
  • 22. travel through time by default time kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael
  • 23. travel through time time kn Davy ow David Davy s kn ow checkpoint s current Peter Peter kn ow s Michael Michael fg.setCheckpointTime(checkpoint);
  • 25. time-scoped iteration t1 t2 change Davy Davy’
  • 26. time-scoped iteration t1 t2 t3 change change Davy Davy’ Davy’’
  • 27. time-scoped iteration t1 t2 t3 tcurrrent change change change Davy Davy’ Davy’’ Davy’’’
  • 28. time-scoped iteration t1 t2 t3 tcurrrent change change change Davy Davy’ Davy’’ Davy’’’ ➡ how to find the version of the vertex you are interested in?
  • 29. time-scoped iteration t1 t2 t3 tcurrrent Davy Davy’ Davy’’ Davy’’’
  • 30. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous
  • 31. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion();
  • 32. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion(); Iterable<Vertex> allDavy = davy.getNextVersions();
  • 33. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion(); Iterable<Vertex> allDavy = davy.getNextVersions(); Iterable<Vertex> selDavy = davy.getPreviousVersions(filter);
  • 34. time-scoped iteration t1 t2 t3 tcurrrent next next next Davy Davy’ Davy’’ Davy’’’ previous previous previous Vertex previousDavy = davy.getPreviousVersion(); Iterable<Vertex> allDavy = davy.getNextVersions(); Iterable<Vertex> selDavy = davy.getPreviousVersions(filter); Interval valid = davy.getTimerInterval();
  • 35. time-scoped iteration ➡ When does an element change?
  • 36. time-scoped iteration ➡ When does an element change? ➡ vertex: ★ setting or removing a property ★ add or remove it from an edge ★ being removed
  • 37. time-scoped iteration ➡ When does an element change? ➡ vertex: ➡ edge: ★ setting or removing a property ★ setting or removing a property ★ add or remove it from an edge ★ being removed ★ being removed
  • 38. time-scoped iteration ➡ When does an element change? ➡ vertex: ➡ edge: ★ setting or removing a property ★ setting or removing a property ★ add or remove it from an edge ★ being removed ★ being removed ➡ ... and each element is time-scoped!
  • 39. temporal graph comparison David Davy Davy kn kn ow ow s s Peter what changed? Peter kn ow s Michael Michael current checkpoint
  • 40. temporal graph comparison ➡ difference (A , B) = union (A , B) - B ➡ ... as a (immutable) graph!
  • 41. temporal graph comparison ➡ difference (A , B) = union (A , B) - B ➡ ... as a (immutable) graph!
  • 42. temporal graph comparison ➡ difference (A , B) = union (A , B) - B ➡ ... as a (immutable) graph! David difference ( , )= kn ow s
  • 43. FluxGraph ... ➡ available on github http://github.com/datablend/fluxgraph
  • 44. use case: longitudinal patient data t1 t2 t3 t4 t5 smoking smoking death patient patient patient patient patient cancer cancer
  • 45. use case: longitudinal patient data ➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)
  • 46. use case: longitudinal patient data ➡ historical data for 15.000 patients over a period of 10 years (2001- 2010) ➡ example analysis: ★ if a male patient is no longer smoking in 2005 ★ what are the chances of getting lung cancer in 2010, comparing patients that smoked before 2005 patients that never smoked
  • 47. use case: longitudinal patient data ➡ get all male non-smokers in 2005 fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
  • 48. use case: longitudinal patient data ➡ get all male non-smokers in 2005 fg.setCheckpointTime(new DateTime(2005,12,31).toDate()); Iterator<Vertex> males = fg.getVertices("gender", "male").iterator()
  • 49. use case: longitudinal patient data ➡ get all male non-smokers in 2005 fg.setCheckpointTime(new DateTime(2005,12,31).toDate()); Iterator<Vertex> males = fg.getVertices("gender", "male").iterator() while (males.hasNext()) { Vertex p2005 = males.next(); boolean smoking2005 = p2005.getEdges(OUT,"smokingStatus").iterator().hasNext(); }
  • 50. use case: longitudinal patient data ➡ which patients were smoking before 2005? boolean smokingBefore2005 = ((FluxVertex)p2005).getPreviousVersions(new TimeAwareFilter() { public TimeAwareElement filter(TimeAwareVertex element) { return element.getEdges(OUT, "smokingStatus").iterator().hasNext() ? element : null; } }).iterator().hasNext();
  • 51. use case: longitudinal patient data ➡ which patients have cancer in 2010 working set of smokers Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate());
  • 52. use case: longitudinal patient data ➡ which patients have cancer in 2010 working set of smokers Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate()); ➡ extract the patients that have an edge to the cancer node
  • 54. gephi plugin for fluxgraph 2010
  • 55. gephi plugin for fluxgraph 2001
  • 56. gephi plugin for blueprints! 1 ➡ available on github http://github.com/datablend/gephi-blueprints-plugin ➡ Support for neo4j, orientdb, dex, rexter, ... 1. Kudos to Timmy Storms (@timmystorms)