SlideShare une entreprise Scribd logo
1  sur  22
SotM-US 2015
Osmose QA
Issue reported
at NYC on 2015-06-07
Frédéric Rodrigo <frederic@carte-libre.fr>
(c)left 2015 - CC-BY-SA v3.0
http://osmose.openstreetmap.fr
Osmose QA ?
Quality tool
Data analysis
Issues detection with rules
Missing data from OpenData
Integration suggestions
Osmose QA way
No duplicate analyses with other tools
No force mapping way
– Adapt to local mapping approach
– No choose one alternative among others
Frontend - Interface
1 ligne 2 ligne 3 ligne 4 ligne
0
2
4
6
8
10
12
1 colonne
2 colonne
3 colonne
Frontend
Issues map
Filter: severity, topic, fixable, items
– Responsive, filter for survey
Coverage map, heatmap
Connection Oauth via osm.org
Export issues: GPX, remote JOSM, Json, RSS…
Issues dumps: SQL, CSV
Frontend - Popup
1. Description
2. Suspects elements
3. Tags
4. Zone links
5. Buttons
6. Suggested fixs
7. Status change: corrected, false positive
Frontend – Tags Editor
Make change into OSM from Osmose QA
Edit only tags
Can apply « fix »
– A nice differential
tags editor
Backend - Frontend
Backend Frontend
OSM Data
API
Backend - Countries
Split : specifics analyser
– Languages, Country, Projection
→ allow statistics
Sublevels split : France, Italy, Belgium, USA, Canada...
70% of planet data
→ Need more servers
→ Aim self support from local community: mapping practice,
knowledge, OpenData
Backend - Analysers
Analysers
– Rules on one object tags
● Capitals, ref...
– SQL spatial rules or multi-objects check
● Reverse roundabout, unconnected highway...
– OpenData features checking
● Schools, post offices, bus stops...
More than 200 analysers
Sax analyser
class Name_Spaces(Plugin):
def init(self, logger):
self.errors[903] = { "item": 5010, "level": 2, "tag": ["name", "fix:chair"], "desc": T_(u"Too
many spaces") }
def node(self, data, tags):
name = tags[u"name"]
if u" " in name:
err.append({"class": 903, "subclass": 0, "text": T_("Name contains successive
spaces")})
if len(err) > 0:
name = re.sub(r' +', ' ', name.strip())
for e in err:
e["fix"] = {"name": name}
issue definition
issue detection
issue possible fix
Osmosis analyserSql10 = """
SELECT
id,
ST_AsText(way_locate(linestring))
FROM
ways
WHERE
tags ? 'junction' AND tags->'junction' = 'roundabout' AND
is_polygon AND ST_IsSimple(linestring) AND
{1} ST_OrderingEquals(ST_Makepolygon(linestring), st_forceRHR(ST_Makepolygon(linestring)))"""
class Analyser_Osmosis_Roundabout_Reverse(Analyser_Osmosis):
def __init__(self, config, logger = None):
Analyser_Osmosis.__init__(self, config, logger)
self.classs_change[1] = {"item":"1050", "level": 1, "tag": ["highway", "roundabout", "fix:chair"], "desc": T_(u"Reverse
roundabout") }
self.callback10 = lambda res: {"class":1, "data":[self.way_full, self.positionAsText]}
if self.config.options.get("driving_side") == "left":
self.driving_side = "NOT "
else:
self.driving_side = ""
def analyser_osmosis_all(self):
self.run(sql10.format("", self.driving_side), self.callback10)
issue detection
issue definition
OpenData analyser
class Analyser_Merge_Poste_FR(Analyser_Merge):
def __init__(self, config, logger = None):
self.missing_official = {"item":"8020", "class": 1, "level": 3, "tag": ["merge", "post"], "desc": T_(u"Post office not integrated") }
self.missing_osm = {"item":"7050", "class": 2, "level": 3, "tag": ["merge", "post"], "desc": T_(u"Post office without ref:FR:LaPoste") }
self.possible_merge = {"item":"8021", "class": 3, "level": 3, "tag": ["merge", "post"], "desc": T_(u"Post office, integration suggestion") }
Analyser_Merge.__init__(self, config, logger,
Source(
name = u"points de contact du réseau postal français",
file = "poste_FR.csv.bz2",
encoding = "ISO-8859-15",
csv = CSV(separator = ";")),
Load("Longitude", "Latitude", table = "poste_fr"),
Mapping(
select = Select(
types = ["nodes", "ways"],
tags = {"amenity": "post_office"}),
osmRef = "ref:FR:LaPoste",
conflationDistance = 1000,
generate = Generate(
static = {
"amenity": "post_office",
"operator": "La Poste",
"source": "data.gouv.fr:LaPoste - 01/2013"},
mapping = {
"ref:FR:LaPoste": "Identifiant",
"name": lambda res: re.sub(self.Principal, " Principal", res[u"Libellé du site"]),
"addr:postcode": "Code postal"} )))
issue definitions
OpenData source file description
OSM data pick-up
Join OpenData & OSM
Map OpenData columns to OSM tags
15 Millions of issues
USA
Germany
World, level=1
France, level=1
Corrections
Errors
New
Rules
USA, level=1
Germany, level=1
Stats
15 millions of issues
– 850 000 of level=1
3 700 000 executed analyses
– 12 000 analysis / 2 days
15 virtuals serveurs
500 rules checked
19,8 / 28 Go : 70 % of planet
352 area (countries or sub areas)
Maproulette
Bridge to Maproulette
Reports choose:
– [world] Highway intersecting building
– [world] Object tagged twice as node or way
– [france] Missing soccer pitch
– [world] Broken highway level continuity
→ OSM as a Marathon
Osmose
http://osmose.openstreetmap.fr
http://wiki.osm.org/Osmose
@osmose_qa

Contenu connexe

Similaire à 20150607 sotm-us-osmose-qa

Osmose, quality assurance tool
Osmose, quality assurance toolOsmose, quality assurance tool
Osmose, quality assurance toolFrédéric Rodrigo
 
Building an ML Platform with Ray and MLflow
Building an ML Platform with Ray and MLflowBuilding an ML Platform with Ray and MLflow
Building an ML Platform with Ray and MLflowDatabricks
 
Nodejs性能分析优化和分布式设计探讨
Nodejs性能分析优化和分布式设计探讨Nodejs性能分析优化和分布式设计探讨
Nodejs性能分析优化和分布式设计探讨flyinweb
 
Megamodeling of Complex, Distributed, Heterogeneous CPS Systems
Megamodeling of Complex, Distributed, Heterogeneous CPS SystemsMegamodeling of Complex, Distributed, Heterogeneous CPS Systems
Megamodeling of Complex, Distributed, Heterogeneous CPS SystemsEugenio Villar
 
Lab 2: Classification and Regression Prediction Models, training and testing ...
Lab 2: Classification and Regression Prediction Models, training and testing ...Lab 2: Classification and Regression Prediction Models, training and testing ...
Lab 2: Classification and Regression Prediction Models, training and testing ...Yao Yao
 
【論文紹介】Relay: A New IR for Machine Learning Frameworks
【論文紹介】Relay: A New IR for Machine Learning Frameworks【論文紹介】Relay: A New IR for Machine Learning Frameworks
【論文紹介】Relay: A New IR for Machine Learning FrameworksTakeo Imai
 
Intravert Server side processing for Cassandra
Intravert Server side processing for CassandraIntravert Server side processing for Cassandra
Intravert Server side processing for CassandraEdward Capriolo
 
NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"
NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"
NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"DataStax Academy
 
What's Coming Next in Sencha Frameworks
What's Coming Next in Sencha FrameworksWhat's Coming Next in Sencha Frameworks
What's Coming Next in Sencha FrameworksGrgur Grisogono
 
Hierarchical free monads and software design in fp
Hierarchical free monads and software design in fpHierarchical free monads and software design in fp
Hierarchical free monads and software design in fpAlexander Granin
 
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...Lucidworks
 
Adventures in civic hacking
Adventures in civic hackingAdventures in civic hacking
Adventures in civic hackingosfameron
 
"Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin "Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin Vasil Remeniuk
 
More Than po: Debugging in LLDB @ CocoaConf SJ 2015
More Than po: Debugging in LLDB @ CocoaConf SJ 2015More Than po: Debugging in LLDB @ CocoaConf SJ 2015
More Than po: Debugging in LLDB @ CocoaConf SJ 2015Michele Titolo
 
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)Large Scale Log Analytics with Solr (from Lucene Revolution 2015)
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)Sematext Group, Inc.
 
SVCC 2013 D3.js Presentation (10/05/2013)
SVCC 2013 D3.js Presentation (10/05/2013)SVCC 2013 D3.js Presentation (10/05/2013)
SVCC 2013 D3.js Presentation (10/05/2013)Oswald Campesato
 
Eclipse Con 2015: Codan - a C/C++ Code Analysis Framework for CDT
Eclipse Con 2015: Codan - a C/C++ Code Analysis Framework for CDTEclipse Con 2015: Codan - a C/C++ Code Analysis Framework for CDT
Eclipse Con 2015: Codan - a C/C++ Code Analysis Framework for CDTElena Laskavaia
 

Similaire à 20150607 sotm-us-osmose-qa (20)

Osmose, quality assurance tool
Osmose, quality assurance toolOsmose, quality assurance tool
Osmose, quality assurance tool
 
huhu
huhuhuhu
huhu
 
Building an ML Platform with Ray and MLflow
Building an ML Platform with Ray and MLflowBuilding an ML Platform with Ray and MLflow
Building an ML Platform with Ray and MLflow
 
Nodejs性能分析优化和分布式设计探讨
Nodejs性能分析优化和分布式设计探讨Nodejs性能分析优化和分布式设计探讨
Nodejs性能分析优化和分布式设计探讨
 
Megamodeling of Complex, Distributed, Heterogeneous CPS Systems
Megamodeling of Complex, Distributed, Heterogeneous CPS SystemsMegamodeling of Complex, Distributed, Heterogeneous CPS Systems
Megamodeling of Complex, Distributed, Heterogeneous CPS Systems
 
Lab 2: Classification and Regression Prediction Models, training and testing ...
Lab 2: Classification and Regression Prediction Models, training and testing ...Lab 2: Classification and Regression Prediction Models, training and testing ...
Lab 2: Classification and Regression Prediction Models, training and testing ...
 
【論文紹介】Relay: A New IR for Machine Learning Frameworks
【論文紹介】Relay: A New IR for Machine Learning Frameworks【論文紹介】Relay: A New IR for Machine Learning Frameworks
【論文紹介】Relay: A New IR for Machine Learning Frameworks
 
Intravert Server side processing for Cassandra
Intravert Server side processing for CassandraIntravert Server side processing for Cassandra
Intravert Server side processing for Cassandra
 
NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"
NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"
NYC* 2013 - "Advanced Data Processing: Beyond Queries and Slices"
 
What's Coming Next in Sencha Frameworks
What's Coming Next in Sencha FrameworksWhat's Coming Next in Sencha Frameworks
What's Coming Next in Sencha Frameworks
 
Hierarchical free monads and software design in fp
Hierarchical free monads and software design in fpHierarchical free monads and software design in fp
Hierarchical free monads and software design in fp
 
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...
Large Scale Log Analytics with Solr: Presented by Rafał Kuć & Radu Gheorghe, ...
 
Adventures in civic hacking
Adventures in civic hackingAdventures in civic hacking
Adventures in civic hacking
 
"Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin "Scala in Goozy", Alexey Zlobin
"Scala in Goozy", Alexey Zlobin
 
More Than po: Debugging in LLDB @ CocoaConf SJ 2015
More Than po: Debugging in LLDB @ CocoaConf SJ 2015More Than po: Debugging in LLDB @ CocoaConf SJ 2015
More Than po: Debugging in LLDB @ CocoaConf SJ 2015
 
AngularJS and SPA
AngularJS and SPAAngularJS and SPA
AngularJS and SPA
 
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)Large Scale Log Analytics with Solr (from Lucene Revolution 2015)
Large Scale Log Analytics with Solr (from Lucene Revolution 2015)
 
Svcc 2013-d3
Svcc 2013-d3Svcc 2013-d3
Svcc 2013-d3
 
SVCC 2013 D3.js Presentation (10/05/2013)
SVCC 2013 D3.js Presentation (10/05/2013)SVCC 2013 D3.js Presentation (10/05/2013)
SVCC 2013 D3.js Presentation (10/05/2013)
 
Eclipse Con 2015: Codan - a C/C++ Code Analysis Framework for CDT
Eclipse Con 2015: Codan - a C/C++ Code Analysis Framework for CDTEclipse Con 2015: Codan - a C/C++ Code Analysis Framework for CDT
Eclipse Con 2015: Codan - a C/C++ Code Analysis Framework for CDT
 

Plus de Frédéric Rodrigo

Cartography: Vector Tiles From an Open Initiative To an Industry Standard
Cartography: Vector Tiles  From an Open Initiative To an Industry StandardCartography: Vector Tiles  From an Open Initiative To an Industry Standard
Cartography: Vector Tiles From an Open Initiative To an Industry StandardFrédéric Rodrigo
 
Osmose-QA, OpenData, Mapillary and MapCSS
Osmose-QA, OpenData, Mapillary and MapCSSOsmose-QA, OpenData, Mapillary and MapCSS
Osmose-QA, OpenData, Mapillary and MapCSSFrédéric Rodrigo
 
Automatic Enhancement of Pedestrian Route using extracted Landmarks from OSM
Automatic Enhancement of Pedestrian Route using extracted Landmarks from OSMAutomatic Enhancement of Pedestrian Route using extracted Landmarks from OSM
Automatic Enhancement of Pedestrian Route using extracted Landmarks from OSMFrédéric Rodrigo
 
Annoter automatiquement un itinéraire piéton avec des repères issus d'OSM
Annoter automatiquement un itinéraire piéton avec des repères issus d'OSMAnnoter automatiquement un itinéraire piéton avec des repères issus d'OSM
Annoter automatiquement un itinéraire piéton avec des repères issus d'OSMFrédéric Rodrigo
 
Osmose-QA, Qualité et intégration de données
Osmose-QA, Qualité et intégration de donnéesOsmose-QA, Qualité et intégration de données
Osmose-QA, Qualité et intégration de donnéesFrédéric Rodrigo
 
OSRM - Open Source Routing Machine
OSRM - Open Source Routing MachineOSRM - Open Source Routing Machine
OSRM - Open Source Routing MachineFrédéric Rodrigo
 
Addok, BAN et BANO dans un bateau
Addok, BAN et BANO dans un bateauAddok, BAN et BANO dans un bateau
Addok, BAN et BANO dans un bateauFrédéric Rodrigo
 
Osmose : la conquête du monde
Osmose : la conquête du mondeOsmose : la conquête du monde
Osmose : la conquête du mondeFrédéric Rodrigo
 
5/5 Osm 20141118-l2.3-réutilisation
5/5 Osm 20141118-l2.3-réutilisation5/5 Osm 20141118-l2.3-réutilisation
5/5 Osm 20141118-l2.3-réutilisationFrédéric Rodrigo
 
4/5 Osm 20141118-l2.2-collecte et contribution
4/5 Osm 20141118-l2.2-collecte et contribution4/5 Osm 20141118-l2.2-collecte et contribution
4/5 Osm 20141118-l2.2-collecte et contributionFrédéric Rodrigo
 
3/5 Osm 20141118-l2.1-être à l'aise avec open streetmap
3/5 Osm 20141118-l2.1-être à l'aise avec open streetmap3/5 Osm 20141118-l2.1-être à l'aise avec open streetmap
3/5 Osm 20141118-l2.1-être à l'aise avec open streetmapFrédéric Rodrigo
 
2/5 Osm 20141106-l1.2-initiation à la contribution
2/5 Osm 20141106-l1.2-initiation à la contribution2/5 Osm 20141106-l1.2-initiation à la contribution
2/5 Osm 20141106-l1.2-initiation à la contributionFrédéric Rodrigo
 
1/5 Osm 20141106-l1.1-présentation
1/5 Osm 20141106-l1.1-présentation1/5 Osm 20141106-l1.1-présentation
1/5 Osm 20141106-l1.1-présentationFrédéric Rodrigo
 

Plus de Frédéric Rodrigo (20)

OSXP 2021 - MAKINA MAPS
OSXP 2021 - MAKINA MAPSOSXP 2021 - MAKINA MAPS
OSXP 2021 - MAKINA MAPS
 
Cartography: Vector Tiles From an Open Initiative To an Industry Standard
Cartography: Vector Tiles  From an Open Initiative To an Industry StandardCartography: Vector Tiles  From an Open Initiative To an Industry Standard
Cartography: Vector Tiles From an Open Initiative To an Industry Standard
 
Osmose-QA, OpenData, Mapillary and MapCSS
Osmose-QA, OpenData, Mapillary and MapCSSOsmose-QA, OpenData, Mapillary and MapCSS
Osmose-QA, OpenData, Mapillary and MapCSS
 
Automatic Enhancement of Pedestrian Route using extracted Landmarks from OSM
Automatic Enhancement of Pedestrian Route using extracted Landmarks from OSMAutomatic Enhancement of Pedestrian Route using extracted Landmarks from OSM
Automatic Enhancement of Pedestrian Route using extracted Landmarks from OSM
 
Annoter automatiquement un itinéraire piéton avec des repères issus d'OSM
Annoter automatiquement un itinéraire piéton avec des repères issus d'OSMAnnoter automatiquement un itinéraire piéton avec des repères issus d'OSM
Annoter automatiquement un itinéraire piéton avec des repères issus d'OSM
 
Osmose-QA, Qualité et intégration de données
Osmose-QA, Qualité et intégration de donnéesOsmose-QA, Qualité et intégration de données
Osmose-QA, Qualité et intégration de données
 
OSRM, Utilisation avancée
OSRM, Utilisation avancéeOSRM, Utilisation avancée
OSRM, Utilisation avancée
 
Open Traffic
Open TrafficOpen Traffic
Open Traffic
 
Osmose-QA
Osmose-QAOsmose-QA
Osmose-QA
 
OSRM L'état du routage
OSRM L'état du routageOSRM L'état du routage
OSRM L'état du routage
 
OSRM - Open Source Routing Machine
OSRM - Open Source Routing MachineOSRM - Open Source Routing Machine
OSRM - Open Source Routing Machine
 
Addok, BAN et BANO dans un bateau
Addok, BAN et BANO dans un bateauAddok, BAN et BANO dans un bateau
Addok, BAN et BANO dans un bateau
 
Mapotempo
MapotempoMapotempo
Mapotempo
 
Osmose : la conquête du monde
Osmose : la conquête du mondeOsmose : la conquête du monde
Osmose : la conquête du monde
 
5/5 Osm 20141118-l2.3-réutilisation
5/5 Osm 20141118-l2.3-réutilisation5/5 Osm 20141118-l2.3-réutilisation
5/5 Osm 20141118-l2.3-réutilisation
 
4/5 Osm 20141118-l2.2-collecte et contribution
4/5 Osm 20141118-l2.2-collecte et contribution4/5 Osm 20141118-l2.2-collecte et contribution
4/5 Osm 20141118-l2.2-collecte et contribution
 
3/5 Osm 20141118-l2.1-être à l'aise avec open streetmap
3/5 Osm 20141118-l2.1-être à l'aise avec open streetmap3/5 Osm 20141118-l2.1-être à l'aise avec open streetmap
3/5 Osm 20141118-l2.1-être à l'aise avec open streetmap
 
2/5 Osm 20141106-l1.2-initiation à la contribution
2/5 Osm 20141106-l1.2-initiation à la contribution2/5 Osm 20141106-l1.2-initiation à la contribution
2/5 Osm 20141106-l1.2-initiation à la contribution
 
1/5 Osm 20141106-l1.1-présentation
1/5 Osm 20141106-l1.1-présentation1/5 Osm 20141106-l1.1-présentation
1/5 Osm 20141106-l1.1-présentation
 
Osm 20141112-montesquieu
Osm 20141112-montesquieuOsm 20141112-montesquieu
Osm 20141112-montesquieu
 

Dernier

Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEarley Information Science
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024The Digital Insurer
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxKatpro Technologies
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationSafe Software
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Igalia
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...apidays
 

Dernier (20)

Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024Tata AIG General Insurance Company - Insurer Innovation Award 2024
Tata AIG General Insurance Company - Insurer Innovation Award 2024
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptxFactors to Consider When Choosing Accounts Payable Services Providers.pptx
Factors to Consider When Choosing Accounts Payable Services Providers.pptx
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 

20150607 sotm-us-osmose-qa

  • 1. SotM-US 2015 Osmose QA Issue reported at NYC on 2015-06-07 Frédéric Rodrigo <frederic@carte-libre.fr> (c)left 2015 - CC-BY-SA v3.0 http://osmose.openstreetmap.fr
  • 2. Osmose QA ? Quality tool Data analysis Issues detection with rules Missing data from OpenData Integration suggestions
  • 3. Osmose QA way No duplicate analyses with other tools No force mapping way – Adapt to local mapping approach – No choose one alternative among others
  • 4. Frontend - Interface 1 ligne 2 ligne 3 ligne 4 ligne 0 2 4 6 8 10 12 1 colonne 2 colonne 3 colonne
  • 5. Frontend Issues map Filter: severity, topic, fixable, items – Responsive, filter for survey Coverage map, heatmap Connection Oauth via osm.org Export issues: GPX, remote JOSM, Json, RSS… Issues dumps: SQL, CSV
  • 6. Frontend - Popup 1. Description 2. Suspects elements 3. Tags 4. Zone links 5. Buttons 6. Suggested fixs 7. Status change: corrected, false positive
  • 7. Frontend – Tags Editor Make change into OSM from Osmose QA Edit only tags Can apply « fix » – A nice differential tags editor
  • 8.
  • 9. Backend - Frontend Backend Frontend OSM Data API
  • 10. Backend - Countries Split : specifics analyser – Languages, Country, Projection → allow statistics Sublevels split : France, Italy, Belgium, USA, Canada... 70% of planet data → Need more servers → Aim self support from local community: mapping practice, knowledge, OpenData
  • 11. Backend - Analysers Analysers – Rules on one object tags ● Capitals, ref... – SQL spatial rules or multi-objects check ● Reverse roundabout, unconnected highway... – OpenData features checking ● Schools, post offices, bus stops... More than 200 analysers
  • 12. Sax analyser class Name_Spaces(Plugin): def init(self, logger): self.errors[903] = { "item": 5010, "level": 2, "tag": ["name", "fix:chair"], "desc": T_(u"Too many spaces") } def node(self, data, tags): name = tags[u"name"] if u" " in name: err.append({"class": 903, "subclass": 0, "text": T_("Name contains successive spaces")}) if len(err) > 0: name = re.sub(r' +', ' ', name.strip()) for e in err: e["fix"] = {"name": name} issue definition issue detection issue possible fix
  • 13. Osmosis analyserSql10 = """ SELECT id, ST_AsText(way_locate(linestring)) FROM ways WHERE tags ? 'junction' AND tags->'junction' = 'roundabout' AND is_polygon AND ST_IsSimple(linestring) AND {1} ST_OrderingEquals(ST_Makepolygon(linestring), st_forceRHR(ST_Makepolygon(linestring)))""" class Analyser_Osmosis_Roundabout_Reverse(Analyser_Osmosis): def __init__(self, config, logger = None): Analyser_Osmosis.__init__(self, config, logger) self.classs_change[1] = {"item":"1050", "level": 1, "tag": ["highway", "roundabout", "fix:chair"], "desc": T_(u"Reverse roundabout") } self.callback10 = lambda res: {"class":1, "data":[self.way_full, self.positionAsText]} if self.config.options.get("driving_side") == "left": self.driving_side = "NOT " else: self.driving_side = "" def analyser_osmosis_all(self): self.run(sql10.format("", self.driving_side), self.callback10) issue detection issue definition
  • 14. OpenData analyser class Analyser_Merge_Poste_FR(Analyser_Merge): def __init__(self, config, logger = None): self.missing_official = {"item":"8020", "class": 1, "level": 3, "tag": ["merge", "post"], "desc": T_(u"Post office not integrated") } self.missing_osm = {"item":"7050", "class": 2, "level": 3, "tag": ["merge", "post"], "desc": T_(u"Post office without ref:FR:LaPoste") } self.possible_merge = {"item":"8021", "class": 3, "level": 3, "tag": ["merge", "post"], "desc": T_(u"Post office, integration suggestion") } Analyser_Merge.__init__(self, config, logger, Source( name = u"points de contact du réseau postal français", file = "poste_FR.csv.bz2", encoding = "ISO-8859-15", csv = CSV(separator = ";")), Load("Longitude", "Latitude", table = "poste_fr"), Mapping( select = Select( types = ["nodes", "ways"], tags = {"amenity": "post_office"}), osmRef = "ref:FR:LaPoste", conflationDistance = 1000, generate = Generate( static = { "amenity": "post_office", "operator": "La Poste", "source": "data.gouv.fr:LaPoste - 01/2013"}, mapping = { "ref:FR:LaPoste": "Identifiant", "name": lambda res: re.sub(self.Principal, " Principal", res[u"Libellé du site"]), "addr:postcode": "Code postal"} ))) issue definitions OpenData source file description OSM data pick-up Join OpenData & OSM Map OpenData columns to OSM tags
  • 15. 15 Millions of issues USA Germany
  • 20. Stats 15 millions of issues – 850 000 of level=1 3 700 000 executed analyses – 12 000 analysis / 2 days 15 virtuals serveurs 500 rules checked 19,8 / 28 Go : 70 % of planet 352 area (countries or sub areas)
  • 21. Maproulette Bridge to Maproulette Reports choose: – [world] Highway intersecting building – [world] Object tagged twice as node or way – [france] Missing soccer pitch – [world] Broken highway level continuity → OSM as a Marathon

Notes de l'éditeur

  1. http://osmose.openstreetmap.fr/fr/errors/graph.png?item=1070