This document discusses completeness assessment of maps. It defines completeness as how well map details reflect the real world. Completeness is important to identify areas for focus editing and understand community health. The document outlines prior work assessing 80 cities and computing details for 1 billion map tiles worldwide. It proposes establishing a "gold standard" of highly complete tiles rated by multiple mappers to measure quality in other datasets and regions. Early findings show dense tiles often have higher completeness, but some less dense tiles are also high quality, so completeness depends on multiple complex factors. The next step is building a model integrating these factors based on gold standard ratings.

1. How Complete is the Map?
State of the Map Asia
ramya@mapbox.com

2. Completeness assessment
• Tries to answer how much of the real
world details reflect on the map
• Feature level focus

3. Completeness is a relative idea.
A map is never complete!

4. Why is completeness research
important?
• Identify areas to focus editing
• Understand health of mapping
community

5. Prior work

6. 80 cities for assessment

7. From cities to tiles

8. Compute basemap details for
1 billion z15 tiles worldwide

9. Gold Standard
• Benchmark / Baseline for a reliable
reference
• Helps measure quality in other datasets
• Human Stamp of Approval
• Use case: To determine model for
feature level completeness that can be
applied to estimate completeness
elsewhere

10. Gold Standard for
Completeness Assessment
• Identify Gold Standard tiles
• Could be different for each feature
• Start with a simple feature: building

11. Identification of gold standard tiles
for buildings
I. Identify candidate tiles
• Manual selection of candidate tiles likely to be
excellent quality
• Based on location (known edit patterns), feature
counts/properties, “popularity”

12. “Popular” = attracts lots of eyeballs

13. Identification of gold standard tiles
for buildings
II. Three mappers rate each candidate
tile on:
• Coverage
• Quality
• Height
• Building Type
• Address Data

14. Coverage:

15. Coverage:

16. Coverage:

17. Coverage:

18. Early learnings
What predicts completeness?

19. Densest tiles often have
higher completeness

20. Dense buildings in gold standard tile
Kathmandu

21. Dense buildings in gold standard tile
Jakarta

22. Dense buildings in gold standard tile
New York
City

23. Dense buildings in gold standard tile
Los Angeles

24. Dense building tiles in gold standard tile
San Francisco

25. Dense building tiles in gold standard tile
Washington DC

26. But some less dense tiles
are also very good

27. Commercial areas have larger
buildings, so less dense

28. Built area as a better metric?

29. Built area must account for parks,
waterbodies, etc.

30. But outside urban core, built
area still not a predictor

31. Suburb: low built area; quality map

32. Road <-> building
correlation?

33. Residential roads in residential areas

34. Commercial areas are
surrounded by Major Highways

35. Conclusion:
It’s complex!

36. Next steps:
Building model integrating
multiple factors based on gold
standard ratings

37. Dashboard

38. ?