22. ○ Initially curious, immediately overwhelmed
○ Only response mappers
○ Communities meet infrequently
○ Hard to identify potential super users
23. ○ Initially curious, immediately overwhelmed
○ Only response mappers
○ Communities meet infrequently
○ Hard to identify potential super users
○ Weren’t that interested
Hello, I’m Dylan from Devseed in DC. We’re a tech shop of a small group of smart folks who tackle big problems for cool people.
One problem we recently helped tackle was--- TRACKING OSM CHANGES IN REAL TIME
With the American Red Cross, and for the Missingmaps effort.
We’ve heard it talked a lot about, but I’m curious how many people have actively taken part in the project-
So, how many folks here have used OpenStreetMap data?
How many folks have made an edit to the database?
Cool. Well, it’s AMAZING.
It’s an amazingly weird creation that is reliable enough that MOST apps actually utilize it,
Mapbox, Tanagram, _most web maps_ google references it
Crowd sourced, possibly most up-to-date
My favorite recent example comes from Alan Mustard,
US Ambassador to Turkmenistan
Talks about existing roads being worthless
Talks about new construction mesing up google maps
But this tool doesn’t show you where the changes came from
Which is all to say, OSM is amazing for humanitarian efforts beacuse of how rapid it can be to immediate responses
Doctors without borders, USAID, the American Red Cross, need a platform they can go to for recent info--
OSM’s platform provides a really easy to go to database they can all reach too
But they need that basemap
Whether it be sharing an experience tromping around with field papers...
...or spending a few hours helping map an unknown place to help aid humanitarians
Most people are unaware of what OSM is
Explaining it to parents
So the general entrance points are either:
Chance
Working with the data
Hearing about a community of mappers or meetup event
This is a great entrance:
Knowledgeable person to provide guidance
When a strong community runs around and maps,
wisdom gets shared,
Lessons get learned
Allan Mustard’s talk - rogue mapper
When a strong community runs around and maps,
wisdom gets shared,
Lessons get learned
Allan Mustard’s talk - rogue mapper
When a strong community runs around and maps,
wisdom gets shared,
Lessons get learned
Allan Mustard’s talk - rogue mapper
One way would be to check out the pieces that make them up
Because retention isn’t great.
Most are hit n’ run
Which means they make a few edits and never more
Why is that? Well in truth we don’t know from the stats alone. But we have some theories. And we can test these theories by tinkering with the way things work and measuring the impact.
Initially excited to map their hometown
Spent twenty minutes mapping buildings
Zoomed out, realized they only mapped four blocks
It takes a lot of effort, with minimal feedback
Initially excited to map their hometown
Spent twenty minutes mapping buildings
Zoomed out, realized they only mapped four blocks
It takes a lot of effort, with minimal feedback
Maybe they’re response mappers.
Wait for HOT Activations
Inspired by ‘doing good’
Less interested in local work
Sometimes communities just don’t meet that frequently
Maptimes, bi-annually
Most groups get together initially, but don’t really provide any in-person follow ups
During larger events, can be hard to identify those who are adding the most
Someone did a great job, hard to tell
Maybe someone would excel with JSOM
Or excited by anyone facet of OSM
Or maybe they just tagged along
That’s fine too.
But user analytics can help us reduce many of those hurdles.
Currently, in OSM itself you can view your own edits,
And there was a neat tool made by Pascal Nies that gave a good overview of what you’ve done.
But these only look at the individual, and don’t really influence or become impacted by the communities they’re a part of
What people are mapping reveals a lot about what motivates them
So we want to see what people are mapping. As many of you know, we can get data about our contributions from
So we built out Planet Stream.
What people are mapping reveals a lot about what motivates them
So we want to see what people are mapping. As many of you know, we can get data about our contributions from
Changesets!
Once you’ve made your edits, you hit save, and boom, a changeset is made.
It’s beautiful!
Interface view
Code version of a changeset
Same information
Computer legible,
user id
So, we have our changeset. This has a users edits, we should be good to go right?
Not quite. The format OSM provides for changesets aren’t complete.
There’s nowhere in OSM ecosystem right now that provides full changeset info.
To pull that out we need to utilize third party applications,
The first component is Planet.osm.org
Collection of all OSM’s metadata
The details in the changesets we saw earlier are provided here
The second main component is overpass api which we grab augmented diffs
Minutely updated information on the geometries of a given area.
By plugging in the id, we can get the changesets.
Set to purge by a certain hashtag
The second main component is overpass api which we grab augmented diffs
Minutely updated information on the geometries of a given area.
By plugging in the id, we can get the changesets.
We combined these two in a tool to create Planet Stream
Other folks have combined these datasets in different ways, but by using these we can create a stream of data that runs in real time
Now while this tool was made by Development Seed, it’s on the shoulders of giants. The two main collossuses here are
To start, we start grabbing the augmented diffs and put it in a basket with planetstream
Next, we run a comparison of all the changeset id’s
If they match, we smash them together to create rich changesets,
or changesets with both the metadata and geographic data
When we first prototyped out Planet Stream,
Leaderboard that tracks most popular hashtags for a six hour period
Live, but on a repeating clock
Can see that tracks both the metadata, and details about the changeset
Counts which hashtag has the most contributions
...so this is one application of planet stream.
With those rich changesets, we can now grind them through workers to derive statistics.
How many roads are in this changeset
How many buildings?
When did it occur?
...And that stream of data IS Planet Stream, but (that’s for individuals)
With those rich changesets, we can now grind them through workers to derive statistics.
How many roads are in this changeset
How many buildings?
When did it occur?
...And that stream of data IS Planet Stream, but (that’s for individuals)
Through worker scripts (also open), we grab all those contributions made to #MissingMaps and derive
User Statistics
Hashtag Totals
Trending hashtags
& Live Map Data
So let’s see it in use
To help reduce the depression from mapping so little
Break up tasks
HOT does it via squares
We could do it by badges
Provide checkpoints
Immediately rewarded
For missingmaps we did this with a badge system
If someone is exclusively a crisis mapper
If someone maps many, ping when a new is afoot
Try to message to get more involved
Or otherwise encourage to lead their community
We can encourage more frequency by adding elements of competition between groups
Creating group stats
Easily see what they’ve done, how much they’ve helped
For MM
Leaderboard competition, two universities against one another
We can create a leaderboard for an event
Allow for quick referencing
Quickly locate osm profile
Send follow-ups
See in real-time how folks are doing
Can also compare different hashtags or projects
Some less appropriate (sub-groups)
In this case, comparing osmafrica to hotosm-project-2104
And for any tech folks out there, this is what the API looks like before it comes to the pages.
Overpass goes down occasionally.
Great tool 99% of the time, not really built to sustain live feeds
This can mess up our stream of data, as it takes awhile to pedal backwards and reprocess
Some edits get dropped
Improving tools like this, or replicated these functionalities will be key in making the system fool-proof
Overpass goes down occasionally.
Great tool 99% of the time, not really built to sustain live feeds
This can mess up our stream of data, as it takes awhile to pedal backwards and reprocess
Some edits get dropped
Improving tools like this, or replicated these functionalities will be key in making the system fool-proof
Still, we believe this is a great start to deriving real-time user analytics.
All the code is on github
Host your own version!
Comment, like, subscribe,
Please leave issue notes
Whitelisted, so it will work for any # in a comment
If you want to see how we processed the stream,
- OSM Stats live on ARC’s github
- Fork and hack
If you want to see how we processed the stream,
- OSM Stats live on ARC’s github
- Fork and hack
Just to bring it back,
We’re doing this to understand the people that make up OSM
Motivate them
Reward them for the hard work,
ARC - funding the project and planet streams development
Marc for designing and building much of the back end
Pascal Neis for much inspiration