The Narrative Mind team seeks to develop tools to optimize discovery and investigation of communication trends on social media for the US Army Cyber Command (ARCYBER). They have conducted 46 interviews with users, experts, and buyers. Their focus has evolved to identifying narratives in real time based on updated guidance from ARCYBER. They plan to further define what a "narrative" means, research how companies visualize narratives, and expand their network of beneficiaries.

1. Team Narrative Mind
Stanford University

2. Team Info
Sponsor: US Army Cyber Command (ARCYBER)
The Narrative Mind team contains experts in software engineering, social media design, and
web-based information operations (IO). We seek to develop tools that will optimize discovery
and investigation of communication trends on social media.
Weekly Total: 10 interviews
Users: 11
Experts: 31
Buyers: 4
Cumulative Total: 46 interviews

3. All the Hypotheses
Global
Tweet-level
Awareness Response
Adversary IO Org Chart
No baseline for monitoring/aggregating
adversary use of tech; cyber targeting
Automatic Narrative Detection
Language/culture experts aren’t able
to work at scale
Virality Predictor
Understand which narratives are the most salient
Important Event Predictor
Preempt real world events
Counter-Narrative Generator
DoD/Gvt. social media
presence is weak
Persistent ID-Alias tracker
Account bans and multiple aliases across
different networks make IDs hard to track
Poison Link Generator
Monitor who views
adversary dark-web
content
Filesharing Site Scraper
Need more cached information
access
Expedited Content Categorization
Scale of social media makes manual efforts
painful

4. Organizational
Chart
(updated)
Analyst “Fish”
(CW3 or NCO)
MAJ “Kramer”
Reports info to...
COL “Tom”
LTC “Zeke”
Director & Deputy
Assesses threats 1-3yrs out
Proposes CONOP?
Gets approval?
BG Frost
Deputy Commander for Ops
ARCYBER
(service component)
NSA
ADM Rogers
Internal ARCYBER
office -
Uses OTA option
LTG Cardon
BG (O8-level)
JOINT FORCES HQ-CYBER
USCYBERCOM
ADM Rogers
USSTRATCOM
COCOMs
CENTCOM,
AFRICOM,
NORTHCOM
Proposes
concepts
Blurred lines
between the two
Secretary Carter
CYBER-only
Acquisition
process
NSA

5. Hypotheses ❏ Last week’s hypotheses were not relevant to updated problem definition.
❏ Too MVP specific.
Experiments ❏ Weren't able to consistently test any previous hypotheses.
Results ❏ ARCYBER confidently re-focused the problem on “identifying narratives in real time”.
❏ Full-circle evolution.
❏ The problem is not content categorization nor org chart-level awareness nor persistent ID
tracking nor event prediction.
Actions 1. Increased frequency of communication with sponsor.
2. Help ARCYBER define what a “narrative” is.
3. Research how commercial companies are visualizing “narratives”.
4. Continue expanding network to beneficiaries who can answer the above questions.
- PACOM, CENTCOM, NPS CORE Lab
Customer Discovery

6. E.g.
Madison, AL: Pro-Trump
(Positive: 61% ± 5%)
Millbrae, CA: Anti-Trump
(Positive: 11% ± 5%)
MVP - 3 Types of “Narratives” Outputted by Our System
A ranked list of
most popular
content over time
A list of hashtags
that appear with
the same
message with
extreme
frequency.
Positive or
negative opinion of
adversaries
among geographic
or demographic
population.
Narrative Type #1 Narrative Type #2 Narrative Type #3
E.g.
Adversary: Trump Campaign
#NY4Trump
#Trump2016
Could be classified by an
analyst as “New York
Primary Push” or similar.
E.g
Could be classified by an
analyst as “Political
Outsider” or similar.

7. Mission Model Canvas
- Categorize social media posts
and users by content for
monitoring and tactical purposes.
- Understand viral potential of
extremist social media posts in
real-time.
- Track how groups use
technology over time.
- Gnip/Twitter/Facebook
- CrowdFlower,
Samasource, or
Mechanical Turk
- Pre-existing social
media service and micro-
labor aggregators
- Third-party access
platforms for social
media.
-Data visualization (Zoic)
-Content analysis
platforms (Sens.ai,
Leidos)
ARCYBER
-Bg. General (decision
maker)
-MAJ/LTC/COL
(operational plan)
-Analysts/Operators
(actionable insights)
COCOMs
-Generals/Analysts
-Tactical decision
makers
- Optimize workflow for
social media analysts
- Effectively visualize
how groups use
technology as they grow.
- Aggregate known
research?
- Expedite categorization of
social media content.
- Use MechanicalTurk to
crowdsource categorization of
content and users.
- Use design of now-defunct
Palantir Torch as inspiration for
how to present content in a
streamlined manner.
- Force multiplier for intelligence analysts: receive cleaner, pre-categorized data, target the
most import priorities in real time
- Increase throughput to quantify and flag viral content.
- Improve the categorization of unstructured social media data points using crowdsourced
micro-task labor.
??????????
- Architecture that can
support massive
concurrent data
aggregation and
analysis.
- Customized UI
- Testing with analysts
- MechanicalTurk or crowdsourcing labor (microtasks)
- UI Development/Testing with ARCYBER analysts.
- Software Development
- Research aggregation?
- Access to Twitter
firehose or API
- Local language
speaking crowdsourcing
staff.
- Accurate testing for
intercoder reliability
- Individual
Analysts/Operators
- ARCYBER (Bg. General,
LTC, Strategic Intiatives
Group)
- Continued partnership
with crowdsourcing firms,
Beneficiaries
Mission AchievementMission Budget/Costs
Buy-In/Support
Deployment
Value
Proposition
Key Activities
Key Resources
Key Partners

8. Value Proposition Canvas - ARCYBER
Products
& Services
Web/Desktop
Application Update research
Customer
Jobs
- New mission priorities.
- Little familiarity with overall
environment.
- No repository for high-level
learnings
Gains
Pains
Gain
Creators
Pain
Relievers
- New opportunities for cyber-
ops
- Social network graph?
-Illustrates evolution of
networks
- Provides insight for
mapping “online terrain”
Visualize information about
how organizations are
using social media
ARCYBER - Operator/Analyst (Actionable Insights)

9. Value Proposition Canvas - ARCYBER
Products
& Services
Web/Desktop
Application
Aggregate insights
from analysts,
compile into plan
passed to decision
maker
- New mission priorities.
- New social media platforms
- Ability to prioritize focus
Customer
Jobs
Gains
Pains
Gain
Creators
Pain
Relievers
-Faster turnaround time with
better data gained from
information awareness software
and technology
- Clear hierarchy of
priorities
ARCYBER - MAJ/LTC/COL (Operational Plans)
-Better data in informing
operational plans provided
to decision makers
-Reducing uncertainty in
operational planning

10. Value Proposition Canvas - ARCYBER
Products
& Services
Web/Desktop
Application
Act on reports and
plans generated
about various cyber
issues across globe
- Little understanding of
ground-level nuances of
challenges
- Little understanding of
proliferating platforms
Customer
Jobs
Gains
Pains
Gain
Creators
Pain
Relievers
- Organizational chart of
technology broken down by
functional purpose
- Timeline-based viewer
- Social network graph?
-Reduce uncertainty of
decision making
-Add methodological rigor
to ARCYBER’s operations
- Accomplish mission
objectives
ARCYBER - Brigadier General (Decision Maker)

11. Value Proposition Canvas - COCOM
Products
& Services
Web/Desktop
Application
Target audience
analysis, finding
opportunities to
optimize influence
operations
Customer
Jobs
-Loss of visibility on key
communicators
- No repository for high-level
learnings
Gains
Pains
Gain
Creators
Pain
Relievers
-Auto-populated list & real-time
visualization
-Allows more accurate targeting
-Illustrates evolution of
networks
- Provides insight for mapping
“online terrain”
Closure of visibility gap
COCOM - Operator/Analyst (Actionable Insights)

12. Value Proposition Canvas - COCOM
Products
& Services
Web/Desktop
Application
-Long term
operational planning
- Measuring
effectiveness
Customer
Jobs
- Unclear picture of how
some networks evolve,
adapt over time
Gains
Pains
Gain
Creators
Pain
Relievers
- Organizational chart of
technology broken down by
functional purpose
- Visualization of key accounts,
dissemination networks over
time
-Illustrates evolution of
dissemination networks
- Provides insight for
measuring operations,
mapping “online terrain”
Greater awareness of
network evolution improves
planning, indicators of
operational effectiveness
COCOM - Maj/LTC/Col. (O4-O6) (Operational Plans)

13. Value Proposition Canvas - COCOM
Products
& Services
Web/Desktop
Application
- Judge potential
effect and risk of
operations
- Approve/reject
plans
Customer
Jobs
- Less familiarity with
internet as an operational
environment
- Needs high level summary
of how orgs use technology
Gains
Pains
Gain
Creators
Pain
Relievers
- Organizational chart of
technology broken down by
functional purpose
- Timeline-based viewer
-Illustrates evolution of info
dissemination networks
- Provides insight for
mapping “online terrain”
Visualize high-level
information about how
orgs use technology.
COCOM - Brigadier General (O7) (Decision Maker)

14. APPENDIX
APPENDIX