SlideShare une entreprise Scribd logo
1  sur  108
Télécharger pour lire hors ligne
NYC DataWeb
                A platform for Integrating Public Data into NYC.gov




                                     Joel Natividad
Click here for narrated version           TCG
                                  Thursday, June 9, 2011
                                     SemTech 2011
About Me

•   TCG Software

    •   Software Services arm of “The Chatterjee Group”

    •   Several Portfolio companies in Lifesciences, Telecom,
        Aviation, Energy, Real Estate, & Info Technology

•   Headquartered in NYC

•   Delivery Centers in Bangalore, Kolkata & Mumbai

•   Look after Knowledge Engineering Practice of TCG
Background
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
Main Goals
•   stimulate development of apps
    that improve access to info
    and govt transparency,
    and;


•   encourage innovation & the
    creation of new IP with
    commercial potential
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
CROWDSOURCING
CROWDSOURCING

 • Wisdom of the Crowd
 • Self-selecting, motivated developers
 • Bang for the Buck
 • Ignites Entrepreneurship
CROWDSOURCING

•   Challenge:
    Improve Recommendation Algorithm
    by 10%

• Dataset:
                                                      STATISTICS
 • 100 million ratings (training set)   •       just 6 days into contest,
 • Half a million Users                         Cinematch bested by 1%


 • 18 thousand movies                   •       20,000 Teams, 150 countries

                                        •       Entrants:
• Prize:                                    •     Bell Labs
    One million US Dollars
                                            •     Opera Solutions

                                            •     Well-renowned universities
CROWDSOURCING

•   Challenge:
    Improve Recommendation Algorithm
    by 10%

• Dataset:
                                                      STATISTICS
 • 100 million ratings (training set)   •       just 6 days into contest,
 • Half a million Users                         Cinematch bested by 1%


 • 18 thousand movies                   •       20,000 Teams, 150 countries

                                        •       Entrants:
• Prize:                                    •     Bell Labs
    One million US Dollars
                                            •     Opera Solutions

                                            •     Well-renowned universities
CROWDSOURCING
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
• Washington DC CTO - Vivek Kundra
•   First Federal CIO - Vivek Kundra
•   First Federal CIO - Vivek Kundra

•   Open Government Initiative

    •   Recovery.gov

    •   Data.gov

    •   USAspending.gov

    •   IT Dashboard

    •   Performance.gov

    •   Fedspace

    •   Citizen Services Dashboard
•   First Federal CIO - Vivek Kundra

•   Open Government Initiative

    •   Recovery.gov

    •   Data.gov

    •   USAspending.gov

    •   IT Dashboard

    •   Performance.gov

    •   Fedspace

    •   Citizen Services Dashboard
•   First Federal CIO - Vivek Kundra

•   Open Government Initiative

    •   Recovery.gov

    •   Data.gov

    •   USAspending.gov

    •   IT Dashboard

    •   Performance.gov

    •   Fedspace

    •   Citizen Services Dashboard
•   First Federal CIO - Vivek Kundra

•   Open Government Initiative

    •   Recovery.gov

    •   Data.gov

    •   USAspending.gov

    •   IT Dashboard

    •   Performance.gov

    •   Fedspace

    •   Citizen Services Dashboard
•   First Federal CIO - Vivek Kundra

•   Open Government Initiative

    •   Recovery.gov




                          }
    •   Data.gov                     Li fe
                                 S u pp o r
                                            t
    •   USAspending.gov

    •   IT Dashboard

    •   Performance.gov

    •   Fedspace

    •   Citizen Services Dashboard
•   First Federal CIO - Vivek Kundra

           •   Open Government Initiative

               •
                  sh   ed
                   Recovery.gov




                                     }
         e t• sla           o u Li fe
                           t S pp
  B u dg          i lli on
                   Data.gov

            • m
                                   ort
       $ 34 o n    USAspending.gov

fr o m •m i l l i
       $8
                   IT Dashboard

               •   Performance.gov

               •   Fedspace

               •   Citizen Services Dashboard
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
Open Data in NYC




Council Member Gale Brewer
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
$ 500 m i l l i o n ! ! !
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
Wh y $ 500
m i l l i o n? ! ? !
Wh y $ 500
m i l l i o n? ! ? !
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
“Integrated”
Inter-Agency System
Data Integration Alphabet Soup

       JMS         SOA              XS
                                      LT
M OM         EAI




                                B
                           OR
 EJB     SOAP       D A             XML
                   M
                          RPC
       BPM                      PO JO
                   BPEL
Data Integration Alphabet Soup
        JMS       SOA
                             XS
                               LT
   M
       EAI


MO




                             ORB
EJ




                               XM L
    B
    SO
        AP




    BPM       MDA BPEL RPC     PO JO
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
and
              Principles              b io ni
                                                ch




•   Cost Effective (NOT $500 million dollars)

•   Easy to Use (Developers/Publishers/Citizens)

•   based on Open Standards

•   Low Adoption Curve

•   Help Accelerate Open Data Innovation

•   Useable Data Now!
The Next Web of Open Linked Data
         February 2009
Useable Data Now

•   “Beautiful” Website

•   Useable by Developers/Publishers/Citizens

•   based on Open Standards

•   Low Adoption Curve

•   Help Accelerate Open Data Innovation

•   Useable Data Now!
What	
  NYCBigApps	
  Developers	
  
                                    were	
  Doing


                                              Download &
                                              Decipher


                 ETL             Text
              Processes


Siloed Data
                             •   Spend inordinate amount of time interpreting data

                             •   Massaged Data was then staged locally

                             •   Developers kept reinventing the wheel

                             •   Limited Data mashups

                             •   Applications disconnected from NYCDatamine
                                                                               46
There must be a
  Better Way
How it Started

•   Oct 12, 2010 - NYCBigApps 2.0 announced

•   Nov 9, 2010 - NYCBigApps 2.0 kickoff meeting

•   late Nov 2010 - spoke with Revelytix/Spry about
    collaborating

•   early Dec 2010 - started work on NYCDataWeb

•   Jan 26, 2011 ~4:30p - submitted entry
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
What	
  We	
  Did


                            Domain
                            Ontology
                                                      Query &
                                                      Results



                                                                 Cache       Optimizer
              Definitions
                                                                 Re-Writer   Planner
Siloed Data
                                                                 Indexes     Rules




                                       Re-Writer    Optimizer   Mapping
                                                                Ontology
                                       Indexes      Planner                  Rules

                                                                Metadata
                                                                Ontology
                                                                                       51
“Beautiful” Website
       Three dashboards were built
• NYC Agile Analytics (Spry)
• NYCreation (SMW+)
  - visualized SPARQL query results
• NYCmantics (SMW+)
  - NYC datamine explorer
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
What’s Next?
Semantic Gap
Developers




Semantic Gap
?!?



Semantic Gap
3.0
3.0
 Developers
3.0




JumpStart Semantics
3.0
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
The Computer for the 
          rest of us.
Semantics for the 
      rest of us.
Semantics for the 
   REST of us.
Phase 2
         Aug 2011 (Powered by NYCDataWeb)

•   Hide Complexity               •   Open-source
    (Simplicity = Adoption)           collaboration with
                                      vendors & other
•   Incorporate the whole             institutions
    NYC datamine
                                  •   Incorporate the best of
•   Make it easier for                Socrata and data.gov
    Publishers
                                  •   Improved Visualizations
•   Make it easier for
    Developers

•   Make it easier for Citizens
Phase 2
         Aug 2011 (Powered by NYCDataWeb)

•   Hide Complexity               •   Open-source
    (Simplicity = Adoption)           collaboration with
                                      vendors & other
•   Incorporate the whole             institutions
    NYC datamine
                                  •   Incorporate the best of
•   Make it easier for                Socrata and data.gov
    Publishers
                                  •   Improved Visualizations
•   Make it easier for
    Developers                    •   Position NYCDataWeb as
                                      the accelerated data
•   Make it easier for Citizens       mashup platform
Phase 3
            Nov 2011 (NYCBigApps 2011)


•   DataWeb Deployment Framework SMW bundle

•   More Data Sources (Federator - Spinner)

•   Linked Open Data

•   Make it easier STILL for Publishers, Developers
    and Citizens

•   Enable Widespread adoption of NYCDataWeb
    (NYCDataWeb bootcamp)
The	
  Broader	
  Vision


                                    Domain
                                    Ontology
                                                         Query &
                                                         Results


                                                             RDF
                                                                          Ontology
                         NYC
                     Information
                         Web
                                                                                        Partners
                                        RDF RDF
                                                                   RDF


                                                   RDF       RDF


                                    Web
                                   Pages
                                                                            Other
Agency	
  Data	
                                  Sensorss               Triplestores          85
Phase 4
                Post NYC BigApps 2011




•   Multiple solutions powered by NYCDataWeb

•   <Your city/community/company here> DataWeb

•   Help foster a viable ecosystem of Linked Data

•   ... keep standing on the shoulders of giants
Semantic
Web
Hans Rosling shows the best stats
       you've ever seen
           February 2006
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
PUBLIC
PUBLIC
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
We need your help & feedback




  A Platform for Integrating Public Data into NYC.gov

                 Find out more at
  http://knoodl.com/ui/groups/NYC_Homepage
NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC
CREDITS
•   Lego Faceparty picture by RichardAM (http://www.richard-am.net/)
•   Lego Inauguration Pictures from various Flickr Users (sluggobear, Atwater, Dan
    Hontz)
•   Lego Luke looses his Hand by Flickr user wwwayazdotcom
•   Tim Berners-Lee highlight from TED (http://www.ted.com/talks/
    tim_berners_lee_on_the_next_web.html)
•   Hans Rosling highlight from TED (http://www.ted.com/talks/
    hans_rosling_shows_the_best_stats_you_ve_ever_seen.html)
•   FlowerPowerpont2.pptx provided by Anna Rosling Rönnlund of gapminder
•   “Star Wars Gangsta Rap” highlight, SizzlechestXXX
    (http://www.youtube.com/watch?v=Ij4w7ChpuaM)
•   Various screenshots provided by Revelytix, Spry Inc. and TCG Software
    Services

Contenu connexe

En vedette

Smart Cities and Big Open Data
Smart Cities and Big Open DataSmart Cities and Big Open Data
Smart Cities and Big Open DataJoel Natividad
 
Ontodia Overview - Semantics and Wikis panel - SemTech West 2012
Ontodia Overview - Semantics and Wikis panel - SemTech West 2012Ontodia Overview - Semantics and Wikis panel - SemTech West 2012
Ontodia Overview - Semantics and Wikis panel - SemTech West 2012Joel Natividad
 
Effortless Hr Offering Presentation
Effortless Hr Offering PresentationEffortless Hr Offering Presentation
Effortless Hr Offering PresentationEffortlessHr1
 
NYCFacets: Metadata, Extrametadata and Crowdknowing
NYCFacets: Metadata, Extrametadata and CrowdknowingNYCFacets: Metadata, Extrametadata and Crowdknowing
NYCFacets: Metadata, Extrametadata and CrowdknowingJoel Natividad
 
The Next Generation of Open Data
The Next Generation of Open DataThe Next Generation of Open Data
The Next Generation of Open DataJoel Natividad
 
Ejercicios practicos de excel ii
Ejercicios practicos de excel iiEjercicios practicos de excel ii
Ejercicios practicos de excel iiJosé Luis
 
Raw data in, Insights out - CKANcon 2015
Raw data in, Insights out - CKANcon 2015Raw data in, Insights out - CKANcon 2015
Raw data in, Insights out - CKANcon 2015Joel Natividad
 
Open source in government
Open source in governmentOpen source in government
Open source in governmentJoel Natividad
 

En vedette (10)

Smart Cities and Big Open Data
Smart Cities and Big Open DataSmart Cities and Big Open Data
Smart Cities and Big Open Data
 
Ontodia Overview - Semantics and Wikis panel - SemTech West 2012
Ontodia Overview - Semantics and Wikis panel - SemTech West 2012Ontodia Overview - Semantics and Wikis panel - SemTech West 2012
Ontodia Overview - Semantics and Wikis panel - SemTech West 2012
 
Effortless Hr Offering Presentation
Effortless Hr Offering PresentationEffortless Hr Offering Presentation
Effortless Hr Offering Presentation
 
NYC Remapped
NYC RemappedNYC Remapped
NYC Remapped
 
NYCFacets: Metadata, Extrametadata and Crowdknowing
NYCFacets: Metadata, Extrametadata and CrowdknowingNYCFacets: Metadata, Extrametadata and Crowdknowing
NYCFacets: Metadata, Extrametadata and Crowdknowing
 
The Next Generation of Open Data
The Next Generation of Open DataThe Next Generation of Open Data
The Next Generation of Open Data
 
Practica word
Practica wordPractica word
Practica word
 
Ejercicios practicos de excel ii
Ejercicios practicos de excel iiEjercicios practicos de excel ii
Ejercicios practicos de excel ii
 
Raw data in, Insights out - CKANcon 2015
Raw data in, Insights out - CKANcon 2015Raw data in, Insights out - CKANcon 2015
Raw data in, Insights out - CKANcon 2015
 
Open source in government
Open source in governmentOpen source in government
Open source in government
 

Similaire à NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC

Linked_Open_Data_Rome_Netcamp_13
Linked_Open_Data_Rome_Netcamp_13Linked_Open_Data_Rome_Netcamp_13
Linked_Open_Data_Rome_Netcamp_13Michele Piunti
 
Graph tour keynote 2019
Graph tour keynote 2019Graph tour keynote 2019
Graph tour keynote 2019Neo4j
 
Rapid Data Exploration With Hadoop
Rapid Data Exploration With HadoopRapid Data Exploration With Hadoop
Rapid Data Exploration With HadoopPeter Skomoroch
 
Netflix Recommender System : Big Data Case Study
Netflix Recommender System : Big Data Case StudyNetflix Recommender System : Big Data Case Study
Netflix Recommender System : Big Data Case StudyKetan Patil
 
TIBCO Advanced Analytics Meetup (TAAM) - June 2015
TIBCO Advanced Analytics Meetup (TAAM) - June 2015TIBCO Advanced Analytics Meetup (TAAM) - June 2015
TIBCO Advanced Analytics Meetup (TAAM) - June 2015Bipin Singh
 
Big Data Ecosystem for Data-Driven Decision Making
Big Data Ecosystem for Data-Driven Decision MakingBig Data Ecosystem for Data-Driven Decision Making
Big Data Ecosystem for Data-Driven Decision MakingAbzetdin Adamov
 
Agile Data Rationalization for Operational Intelligence
Agile Data Rationalization for Operational IntelligenceAgile Data Rationalization for Operational Intelligence
Agile Data Rationalization for Operational IntelligenceInside Analysis
 
Graphs fun vjug2
Graphs fun vjug2Graphs fun vjug2
Graphs fun vjug2Neo4j
 
Data.gov Open Data Day
Data.gov Open Data DayData.gov Open Data Day
Data.gov Open Data DayJeanne Holm
 
Open Data Briefing for Alameda County Data Sharing Committee
Open Data Briefing for Alameda County Data Sharing CommitteeOpen Data Briefing for Alameda County Data Sharing Committee
Open Data Briefing for Alameda County Data Sharing CommitteeUrban Strategies Council
 
Continuum Analytics and Python
Continuum Analytics and PythonContinuum Analytics and Python
Continuum Analytics and PythonTravis Oliphant
 
Apache Geode - The First Six Months
Apache Geode -  The First Six MonthsApache Geode -  The First Six Months
Apache Geode - The First Six MonthsAnthony Baker
 
Gis - open source potentials
Gis  - open source potentialsGis  - open source potentials
Gis - open source potentialsTim Willoughby
 
Séminaire Big Data Alter Way - Elasticsearch - octobre 2014
Séminaire Big Data Alter Way - Elasticsearch - octobre 2014Séminaire Big Data Alter Way - Elasticsearch - octobre 2014
Séminaire Big Data Alter Way - Elasticsearch - octobre 2014ALTER WAY
 
U of A Web Strategy and Sitecore
U of A Web Strategy and SitecoreU of A Web Strategy and Sitecore
U of A Web Strategy and SitecoreTim Schneider
 
Department of Commerce App Challenge: Big Data Dashboards
Department of Commerce App Challenge: Big Data DashboardsDepartment of Commerce App Challenge: Big Data Dashboards
Department of Commerce App Challenge: Big Data DashboardsBrand Niemann
 
Analytical Innovation: How to Build the Next Generation Data Platform
Analytical Innovation: How to Build the Next Generation Data PlatformAnalytical Innovation: How to Build the Next Generation Data Platform
Analytical Innovation: How to Build the Next Generation Data PlatformVMware Tanzu
 
Anatomy of a Big Data Application (BDA)
Anatomy of a Big Data Application (BDA)Anatomy of a Big Data Application (BDA)
Anatomy of a Big Data Application (BDA)BloomReach
 

Similaire à NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC (20)

Linked_Open_Data_Rome_Netcamp_13
Linked_Open_Data_Rome_Netcamp_13Linked_Open_Data_Rome_Netcamp_13
Linked_Open_Data_Rome_Netcamp_13
 
Graph tour keynote 2019
Graph tour keynote 2019Graph tour keynote 2019
Graph tour keynote 2019
 
Rapid Data Exploration With Hadoop
Rapid Data Exploration With HadoopRapid Data Exploration With Hadoop
Rapid Data Exploration With Hadoop
 
Netflix Recommender System : Big Data Case Study
Netflix Recommender System : Big Data Case StudyNetflix Recommender System : Big Data Case Study
Netflix Recommender System : Big Data Case Study
 
TIBCO Advanced Analytics Meetup (TAAM) - June 2015
TIBCO Advanced Analytics Meetup (TAAM) - June 2015TIBCO Advanced Analytics Meetup (TAAM) - June 2015
TIBCO Advanced Analytics Meetup (TAAM) - June 2015
 
Big Data Ecosystem for Data-Driven Decision Making
Big Data Ecosystem for Data-Driven Decision MakingBig Data Ecosystem for Data-Driven Decision Making
Big Data Ecosystem for Data-Driven Decision Making
 
Agile Data Rationalization for Operational Intelligence
Agile Data Rationalization for Operational IntelligenceAgile Data Rationalization for Operational Intelligence
Agile Data Rationalization for Operational Intelligence
 
Graphs fun vjug2
Graphs fun vjug2Graphs fun vjug2
Graphs fun vjug2
 
BigData.pptx
BigData.pptxBigData.pptx
BigData.pptx
 
Highlights from SharePoint Conference 2011
Highlights from SharePoint Conference 2011Highlights from SharePoint Conference 2011
Highlights from SharePoint Conference 2011
 
Data.gov Open Data Day
Data.gov Open Data DayData.gov Open Data Day
Data.gov Open Data Day
 
Open Data Briefing for Alameda County Data Sharing Committee
Open Data Briefing for Alameda County Data Sharing CommitteeOpen Data Briefing for Alameda County Data Sharing Committee
Open Data Briefing for Alameda County Data Sharing Committee
 
Continuum Analytics and Python
Continuum Analytics and PythonContinuum Analytics and Python
Continuum Analytics and Python
 
Apache Geode - The First Six Months
Apache Geode -  The First Six MonthsApache Geode -  The First Six Months
Apache Geode - The First Six Months
 
Gis - open source potentials
Gis  - open source potentialsGis  - open source potentials
Gis - open source potentials
 
Séminaire Big Data Alter Way - Elasticsearch - octobre 2014
Séminaire Big Data Alter Way - Elasticsearch - octobre 2014Séminaire Big Data Alter Way - Elasticsearch - octobre 2014
Séminaire Big Data Alter Way - Elasticsearch - octobre 2014
 
U of A Web Strategy and Sitecore
U of A Web Strategy and SitecoreU of A Web Strategy and Sitecore
U of A Web Strategy and Sitecore
 
Department of Commerce App Challenge: Big Data Dashboards
Department of Commerce App Challenge: Big Data DashboardsDepartment of Commerce App Challenge: Big Data Dashboards
Department of Commerce App Challenge: Big Data Dashboards
 
Analytical Innovation: How to Build the Next Generation Data Platform
Analytical Innovation: How to Build the Next Generation Data PlatformAnalytical Innovation: How to Build the Next Generation Data Platform
Analytical Innovation: How to Build the Next Generation Data Platform
 
Anatomy of a Big Data Application (BDA)
Anatomy of a Big Data Application (BDA)Anatomy of a Big Data Application (BDA)
Anatomy of a Big Data Application (BDA)
 

Dernier

UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.francesco barbera
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAshyamraj55
 
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdfJamie (Taka) Wang
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataSafe Software
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncObject Automation
 
Babel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxBabel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxYounusS2
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfAnna Loughnan Colquhoun
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceMartin Humpolec
 

Dernier (20)

UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.Digital magic. A small project for controlling smart light bulbs.
Digital magic. A small project for controlling smart light bulbs.
 
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPAAnypoint Code Builder , Google Pub sub connector and MuleSoft RPA
Anypoint Code Builder , Google Pub sub connector and MuleSoft RPA
 
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
20200723_insight_release_plan_v6.pdf20200723_insight_release_plan_v6.pdf
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial DataCloud Revolution: Exploring the New Wave of Serverless Spatial Data
Cloud Revolution: Exploring the New Wave of Serverless Spatial Data
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation Inc
 
Babel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptxBabel Compiler - Transforming JavaScript for All Browsers.pptx
Babel Compiler - Transforming JavaScript for All Browsers.pptx
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdf
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
Things you didn't know you can use in your Salesforce
Things you didn't know you can use in your SalesforceThings you didn't know you can use in your Salesforce
Things you didn't know you can use in your Salesforce
 

NYC Data Web (static version) - A Semantic, Open Public Data Exchange for NYC

  • 1. NYC DataWeb A platform for Integrating Public Data into NYC.gov Joel Natividad Click here for narrated version TCG Thursday, June 9, 2011 SemTech 2011
  • 2. About Me • TCG Software • Software Services arm of “The Chatterjee Group” • Several Portfolio companies in Lifesciences, Telecom, Aviation, Energy, Real Estate, & Info Technology • Headquartered in NYC • Delivery Centers in Bangalore, Kolkata & Mumbai • Look after Knowledge Engineering Practice of TCG
  • 6. Main Goals • stimulate development of apps that improve access to info and govt transparency, and; • encourage innovation & the creation of new IP with commercial potential
  • 10. CROWDSOURCING • Wisdom of the Crowd • Self-selecting, motivated developers • Bang for the Buck • Ignites Entrepreneurship
  • 11. CROWDSOURCING • Challenge: Improve Recommendation Algorithm by 10% • Dataset: STATISTICS • 100 million ratings (training set) • just 6 days into contest, • Half a million Users Cinematch bested by 1% • 18 thousand movies • 20,000 Teams, 150 countries • Entrants: • Prize: • Bell Labs One million US Dollars • Opera Solutions • Well-renowned universities
  • 12. CROWDSOURCING • Challenge: Improve Recommendation Algorithm by 10% • Dataset: STATISTICS • 100 million ratings (training set) • just 6 days into contest, • Half a million Users Cinematch bested by 1% • 18 thousand movies • 20,000 Teams, 150 countries • Entrants: • Prize: • Bell Labs One million US Dollars • Opera Solutions • Well-renowned universities
  • 19. • Washington DC CTO - Vivek Kundra
  • 20. First Federal CIO - Vivek Kundra
  • 21. First Federal CIO - Vivek Kundra • Open Government Initiative • Recovery.gov • Data.gov • USAspending.gov • IT Dashboard • Performance.gov • Fedspace • Citizen Services Dashboard
  • 22. First Federal CIO - Vivek Kundra • Open Government Initiative • Recovery.gov • Data.gov • USAspending.gov • IT Dashboard • Performance.gov • Fedspace • Citizen Services Dashboard
  • 23. First Federal CIO - Vivek Kundra • Open Government Initiative • Recovery.gov • Data.gov • USAspending.gov • IT Dashboard • Performance.gov • Fedspace • Citizen Services Dashboard
  • 24. First Federal CIO - Vivek Kundra • Open Government Initiative • Recovery.gov • Data.gov • USAspending.gov • IT Dashboard • Performance.gov • Fedspace • Citizen Services Dashboard
  • 25. First Federal CIO - Vivek Kundra • Open Government Initiative • Recovery.gov } • Data.gov Li fe S u pp o r t • USAspending.gov • IT Dashboard • Performance.gov • Fedspace • Citizen Services Dashboard
  • 26. First Federal CIO - Vivek Kundra • Open Government Initiative • sh ed Recovery.gov } e t• sla o u Li fe t S pp B u dg i lli on Data.gov • m ort $ 34 o n USAspending.gov fr o m •m i l l i $8 IT Dashboard • Performance.gov • Fedspace • Citizen Services Dashboard
  • 29. Open Data in NYC Council Member Gale Brewer
  • 35. $ 500 m i l l i o n ! ! !
  • 39. Wh y $ 500 m i l l i o n? ! ? !
  • 40. Wh y $ 500 m i l l i o n? ! ? !
  • 49. Data Integration Alphabet Soup JMS SOA XS LT M OM EAI B OR EJB SOAP D A XML M RPC BPM PO JO BPEL
  • 50. Data Integration Alphabet Soup JMS SOA XS LT M EAI MO ORB EJ XM L B SO AP BPM MDA BPEL RPC PO JO
  • 52. and Principles b io ni ch • Cost Effective (NOT $500 million dollars) • Easy to Use (Developers/Publishers/Citizens) • based on Open Standards • Low Adoption Curve • Help Accelerate Open Data Innovation • Useable Data Now!
  • 53. The Next Web of Open Linked Data February 2009
  • 54. Useable Data Now • “Beautiful” Website • Useable by Developers/Publishers/Citizens • based on Open Standards • Low Adoption Curve • Help Accelerate Open Data Innovation • Useable Data Now!
  • 55. What  NYCBigApps  Developers   were  Doing Download & Decipher ETL Text Processes Siloed Data • Spend inordinate amount of time interpreting data • Massaged Data was then staged locally • Developers kept reinventing the wheel • Limited Data mashups • Applications disconnected from NYCDatamine 46
  • 56. There must be a Better Way
  • 57. How it Started • Oct 12, 2010 - NYCBigApps 2.0 announced • Nov 9, 2010 - NYCBigApps 2.0 kickoff meeting • late Nov 2010 - spoke with Revelytix/Spry about collaborating • early Dec 2010 - started work on NYCDataWeb • Jan 26, 2011 ~4:30p - submitted entry
  • 60. What  We  Did Domain Ontology Query & Results Cache Optimizer Definitions Re-Writer Planner Siloed Data Indexes Rules Re-Writer Optimizer Mapping Ontology Indexes Planner Rules Metadata Ontology 51
  • 61. “Beautiful” Website Three dashboards were built • NYC Agile Analytics (Spry) • NYCreation (SMW+) - visualized SPARQL query results • NYCmantics (SMW+) - NYC datamine explorer
  • 85. 3.0
  • 88. 3.0
  • 92. The Computer for the  rest of us.
  • 93. Semantics for the  rest of us.
  • 94. Semantics for the  REST of us.
  • 95. Phase 2 Aug 2011 (Powered by NYCDataWeb) • Hide Complexity • Open-source (Simplicity = Adoption) collaboration with vendors & other • Incorporate the whole institutions NYC datamine • Incorporate the best of • Make it easier for Socrata and data.gov Publishers • Improved Visualizations • Make it easier for Developers • Make it easier for Citizens
  • 96. Phase 2 Aug 2011 (Powered by NYCDataWeb) • Hide Complexity • Open-source (Simplicity = Adoption) collaboration with vendors & other • Incorporate the whole institutions NYC datamine • Incorporate the best of • Make it easier for Socrata and data.gov Publishers • Improved Visualizations • Make it easier for Developers • Position NYCDataWeb as the accelerated data • Make it easier for Citizens mashup platform
  • 97. Phase 3 Nov 2011 (NYCBigApps 2011) • DataWeb Deployment Framework SMW bundle • More Data Sources (Federator - Spinner) • Linked Open Data • Make it easier STILL for Publishers, Developers and Citizens • Enable Widespread adoption of NYCDataWeb (NYCDataWeb bootcamp)
  • 98. The  Broader  Vision Domain Ontology Query & Results RDF Ontology NYC Information Web Partners RDF RDF RDF RDF RDF Web Pages Other Agency  Data   Sensorss Triplestores 85
  • 99. Phase 4 Post NYC BigApps 2011 • Multiple solutions powered by NYCDataWeb • <Your city/community/company here> DataWeb • Help foster a viable ecosystem of Linked Data • ... keep standing on the shoulders of giants
  • 101. Hans Rosling shows the best stats you've ever seen February 2006
  • 103. PUBLIC
  • 104. PUBLIC
  • 106. We need your help & feedback A Platform for Integrating Public Data into NYC.gov Find out more at http://knoodl.com/ui/groups/NYC_Homepage
  • 108. CREDITS • Lego Faceparty picture by RichardAM (http://www.richard-am.net/) • Lego Inauguration Pictures from various Flickr Users (sluggobear, Atwater, Dan Hontz) • Lego Luke looses his Hand by Flickr user wwwayazdotcom • Tim Berners-Lee highlight from TED (http://www.ted.com/talks/ tim_berners_lee_on_the_next_web.html) • Hans Rosling highlight from TED (http://www.ted.com/talks/ hans_rosling_shows_the_best_stats_you_ve_ever_seen.html) • FlowerPowerpont2.pptx provided by Anna Rosling Rönnlund of gapminder • “Star Wars Gangsta Rap” highlight, SizzlechestXXX (http://www.youtube.com/watch?v=Ij4w7ChpuaM) • Various screenshots provided by Revelytix, Spry Inc. and TCG Software Services