An excerpt from a presentation at the 2013 Energy Security Council describing the utility and benefits of the Quorum Engine, an R&D project here at IDV Solutions that detects critical events in the Twitter stream through the confluence of time, place, and topic. Find the descriptions of the service in the speaker notes section of each slide.
The standard practice of monitoring Twitter for risk management requires a person to firstsearch for a term or choose from various categories of security and emergency-response keywords, somehow knowing ahead of time what topic may reveal a threat, and where that might emerge. But one of the benefits of an automated system of listening is the ability to highlight when any word sees a drastic and unusual increase in use, especially when that term is unknown by an operator or off the radar of the team overall.In this way, security operators wouldn’t be subject to a library of pre-determined threat terms or be saddled with the task of manually searching here or there in the hopes of perhaps stumbling on an unfolding event, where it’s happening. Rather, an engine that consumes the never ending blast of tweets and scans it for a critical mass of three factors: time, location, and topic, ensures that important emerging incidents aren’t missed. This three-part tipping point is the signature of an event of interest that is then used to proactively generate alerts to stakeholders and locally effected individuals.
For example, what you see here are the locations of tweets in and around Boston on April 15, 2013. The day of the Boston Marathon. The result is essentially a population map of Boston on that day.
These highlighted tweets, by the way, contain the word, “marathon,” and plot a tight signal of the course for us.
But what do the tweets look like if we simulate for an automated alerting engine that triggers at the intersection of 1, a sudden spike in tweeting, 2, tweets within close proximity to each other, and 3, tweets sharing a common subject –whatever that is.
These are the tweets that are compact in time –a rapid period of twitter activity in the area. This happened to be the moments following the bombing.
Here are those tweets, further isolated to those that were geographically compact, happening in tight proximity to each other.
And those tweets, refined further to only those sharing a common topic.The result is a timely unit of intelligence signals, many including photographs, indicating something of interest has just occurred in a defined location. These can be can be triaged by security and emergency response personnel and vetted for sentiment and relevance perhaps long before standard alerting sources may come in.By the way, among the words that saw the greatest spike in use, compared to the general population of tweets, and a signature of tweets related to the bombing, was the word, “just.” Use of the word “just” saw a 1,300% increase in use in those tweets relating to the sudden explosions. This is an example of mining actual events to reveal surprising word-use signatures that were unlikely to have been predicted in some emergency word dictionary. These signatures then feed back into the alerting engine to improve the scanning process.
The timely digital delivery of alert summaries, with contextual information on the nature and proximity of the threat, to only organizationally-relevant stakeholders and locally affected employees, whether residential or travelling, reduces the problem of false positives and its debilitating fatigue of over-notification.
