Continuous Transformation
Environment
Phase III Implementation,
Prototypes, and T&E-based
Use Cases

Overview
• Phase III Technical Goals
• Initial CTE Technology Focus Areas
• Overview of the CTE Technology ingest
process
• Use-Case Approach to T&E
• GPU Investigations
• Current CTE Use Cases
• Next Steps
2

Phase III Technical Goals
• Establish a large-scale testbed capabilities that enable emulation of
commercial and cloud systems at scale
– Implement a hybrid public/private/multi-tenant cloud
– 1024 public IP addresses – enough to support the largest computing
system built so far
• Begin the CTE system integration with high-speed interfaces
– All the systems have 1G interfaces, will implement 10G, 40G Ethernet and
Fibre Channel by the end of Q3 FY2014
• Integrate:
– All the systems into the central management, provisioning, and
configuration control services
– Machine provisioning with software defined networking (SDN)
• Implement the ability to provision virtual and physical machines
with complex applications and databases pre-loaded
• Enable a vastly simplified complex system/architecture
implementation-integration-interoperability capability
3

Initial CTE Technology Focus Areas
• Establish the central CTE lab infrastructure at the
QTS Richmond facility
• Support connectivity and integration with the
satellite facilities
• Demonstrate high performance facility
integration
• Ingest an initial set of technology partners to
explore a variety of cloud computing
infrastructure hardware and software
• User focused demonstrations
4

CTE Technology Ingest Process
• Technology provider interviews
– Organization and key person assessment
– Technology overview, typical use cases, existing or
envisioned customer base
– Standards compliance
– Open Systems/Open APIs
• Use Case Mapping
• Modifications and/or Interoperability Implementation
• Component/System Integration
• Use Case Test/Demonstration
5

Use-Case Approach to T&E
• Isolated technology tests are of limited value
• Test both the technology/service and the
providing organization
• Provide quantifiable value to participants
– Metrics of various types
– Reference third party installation and test
– Neutral test, integration, and demonstration
capability
– New partners and business opportunities
6

GPU-based Investigations
• HP-nVidia Partnership Testing
• Video and Imagery Analytics
• Computational Astronomy
• Computational Cryptography
• Computational Neuroscience
• Cloud-based Robotics Control and
Management
• Immersive Stereoscopic Visualization

8
CTE Prototypes and Use Cases
Number Name Sponsors Short Description
CTE 13-1 DIY Cloud Infrastructure: Rapid Deployment of a Private and
Public Cloud with Amazon-style Ease-of-Use and Accounting
HP, Convergence HP Demonstration of the Cloud Matrix for IaaS, Virtual Desktop demo with
Convergence user devices
CTE 13-2 Intercontinental, Large-scale VIDINT Command/Analysis Center:
Distribution, Storage, Full Field of View Tracking, and full video
and image stream search with arbitrary image(s)
Pixia, Aspera, Signal Innovations, pixLogic, Flexanalytics,
Cloud Front Group, L3, HP, M3 Com, Ciena, Infinite
Dimensions, Terramajic
Show full field of view tracking and analytics demo with metadata transfer to
Autonomy advanced analytics
CTE 13-3 Tapping the Firehose: Large-scale Social Media Analytics and
Command Center
InTTENSITY, Optensity, Infinite Dimensions Working with data sources such as the main Twitter feed is challenging. A social
media command center will be demonstrated, providing insight into a variety of
topics.
CTE 13-4 Making Big Data Pay: Visual Programming of Advanced Analytics
and Manipulation of Data
Optensity, Flexanalytics, Pixlogic Demonstrate an easy-to-use visual programming tool for advanced analytics.
Integrate Flexanalytics/Pixlogic video analytics.
CTE 13-5 Beyond the OODA Loop: Collect-Correlate-Alert-Act (C2A2) Attivio, Ringtail A demonstration of decision data is drawn from multiple sources, correlated, and
alerts generated and sent for actions to be taken in the context of shipping port
management and security.
CTE 13-6 Multi-INT Enterprise Service "Dial tone" and Collect-Correlate-
Alert-Act
MarkLogic, Ringtail A multi-INT scenario illustrating the Collect-Correlate-Alert-Act (C2A2) paradigm.
CTE 13-7 Next-Generation Cloud-based C4ISR Visualization Ringtail A next-generation command center, used by the Chairman of the Joint Chiefs of
Staff , in terms of both software and hardware.
CTE 13-8 Big Data Storage and Analytics Oracle, Platfora, MapR, Cloudera, Hortonworks, Sqrrl,
IBM, Security First
Data cloud protection, visual tools and analytics for Hadoop-based systems in
multiple facilities
CTE 13-9 Object Storage and Analytics Hitachi Large scale object store and manipulation
CTE 13-10 iCommand Dell, AAI, Textron Next Generation Command Center using Chairman of the Joint Chiefs of Staff
visualization system
CTE 13-11 Video Management KSI Video Social Media System for Video and Image Analysis
CTE 13-12 Asteroid Detection/Computational Astronomy UN Space Programme II, NASA, ESA The CTE supported an international team in an asteroid detection contest. The
CTE team discovered a previously unknown Main Belt asteroid, a rare
achievement.
CTE 14-1 Large-scale Immersive Training and Evaluation Private sector investors, 5 NFL Teams, OSD P&R Integrate government founded software development kits (SDKs), such as the
University of Southern California Institute of Creative Technology's Virtual Human
Toolkit, and create a next-generation immersive training environment, with
improved SDKs.
CTE 14-2 COTS Test and Evaluation NRO and others Formally test COTS technology and provide evaluations to interested government
agencies. NRO has started working with the CTE to evaluate the high-speed
message acceleration hardware employed by large-scale stock and bond trading
organizations.
CTE 14-3 Bitcoin Mining: Computational Cryptography CTE Finding bitcoins, a process refered to as bitcoin mining, is an exercise in
computational cryptography. The CTE is strtudying the algrothms and technology
associated with bitcoin mining as community innovations in gryptography, and
real experience with bitcoins. Bitcoins are used by criminals and others who wish
to obscure financial transactions.
CTE 14-4 Automating Identification FBI, Verizon, Infinite Dimensions Efforts such as the FBI's Next Generation Ientification (NGI) program need to
utilize a multi-source data to ID a variety of entities and concepts. This project
evealuates and integrates technology to provide insight into promising
technologies that are relevant to this task.

DIY Cloud Infrastructure
• Show how easy it was to integrate various
cloud systems through a single and unified
provisioning and cost accounting system
• Provision both virtual and physical machines
(for performance)
• Permits a better match between task and
execution architecture
• Eliminate vendor lock-in
9

Features
• Infrastructure-as-a-Service (IaaS) for private and
hybrid cloud environments
• Self-service infrastructure portal for quick auto-
provisioning
– Integrated billing
• Support of heterogeneous environments
– Cloud-bursting to a variety of public cloud providers
– Supported hypervisors include VMware, Microsoft
HyperV, Red Hat KVM, and Integrity VMs
10

Large-Scale VIDINT, IMINT
“Netflix for the DoD/IC”
• Used the software transport software used by Netflix
to move the video and imagery
• Managed video and imagery using Pixia’s system
• Employed Project Guppy’s long-distance, high
performance networking
– Aspera’s London lab to QTS Richmond
• Integrated:
– Video and imagery analytics on ingest using Signal
Innovation Group’s software to track all moving elements
of a video or image stream
– Video and imagery search using pixLogic’s system
• Integrated into HP Autonomy’s correlation and
alerting engine 11

Performance Metrics
• No observable latency in the transmission
– Phase III hypothesis: Project Guppy team can provide
affordable high performance transoceanic circuits that
can deliver high volume and velocity content
anywhere
• Days for the key integrations:
– The Pixia, SIG, and pixLogic integrations were
facilitated via STANAG 4559, NATO Standard Image
Library Interface
– Open API’s facilitated the integration into HP’s
Autonomy
– Phase III hypothesis: Other video and image analytics,
correlation, and alerting engines can be easily
integrated 12

Large-Scale Social Media Analytics and
Command Center
• Integrates full Twitter Firehose, Facebook, etc.
• Data is injected into a real-time ingestion and
orchestration engine that allows creation of
distinct processing pipelines to filter
• Performs specific Natural Language Processing
(NLP) based enrichment of the social media
data of interest
13

Visual Programming of Advanced Analytics – Video
Analytics Integration
• Rapidly compose and execute data analytics
across multiple clouds
• Graphical composition for analytic workflows,
virtualizes the composed workflows (or Apps)
• Executes the Apps across multiple clouds
• Integrated with piXlogic
– Provides visual search solutions that automatically
analyze and index the contents of images and
video files
14

Beyond the OODA Loop
Collect-Correlate-Alert-Act (C2A2)
• Integrated Tibco’s Spotfire analytics capability
with Ringtail’s Common Operational Picture
visualization use by the Chairman of the Joint
Chiefs of Staff (CJCS)
– Integrated open source data about ship
movement between and in ports to Spotfire
– Correlation calculations triggered Alerts in Spotfire
– Alert messages from Spotfire sent to the CJCS
Command System
15

Performance Metrics
• Spotfire integrated and collected multiple
open source data feeds into the correlation
process
• The Spotfire-CJCS COP integration was
performed in less than one week
• The Java Message Service (JMS) was leveraged
to provide interoperability
• Open APIs also facilitated integration
16

Multi-INT Enterprise Service
"Dialtone" and C2A2
• Experimented with the notion of an enterprise
service registry
• Implemented via the Representational State
Transfer (REST) construct
• Integrated multiple data sources and presented
them to an enterprise search function
– Both unstructured and structured
– Demonstration employed open source government
data
17

Next-Generation Cloud-based
C4ISR Visualization
• A more complete demonstration of Ringtail’s
Common Operational Picture visualization use by
the Chairman of the Joint Chiefs of Staff (CJCS)
• Data and capability integration and interactivity
are supported via:
– Well-defined JMS-based messaging
– REST interface support
• Video and collaboration integration to overall
package was interesting
18

Big Data Storage and Analytics
• Showcased Data Cloud Protect participants
• Included Hadoop vendors and analytics
partners
• Provided a venue to educate the audience on
the operational issues
• Highlighted initial collaborative efforts and
results
19

Object Storage and Analytics
• Installed and implemented Hitachi Data Systems
prototype Object Store highlighting:
– Multiple Entry Points: The system must allow for multiple independent
applications simultaneously performing operations such as read and
write.
– Global Namespace: An object store should present a global
namespace (GNS) to the client.
– Access Protocol: The access protocol should work equally well over a
WAN (such as the Internet) as over a LAN.
• Therefore, the access protocol cannot be chatty, as most network file
system protocols tend to be
• Further, it should support mobile devices, such as smartphones, as clients
– Unstructured Data: The system must be designed to optimize for
unstructured data as this will be the predominant type
– Hardware Agnostic: Hardware changes frequently. This includes
servers, storage subsystems, and even the storage medium
20

Social Media and Video
Management
• Cloud-based video and associated “Big Data”
management services
• Focus is video/data management and sharing
from “Remote Sensing Platforms” (e.g.,
unmanned aircraft, boats, underwater
vehicles)
• Social media-like interface and approach to
content management, and analytics
integration
21

VIDINT Command Center
• COTS-based approach to a Video Intelligence
Command Center
• Focused around cameras and facility physical
security
• Integrates analytics capable of license plate
recognition (LPR) with the camera feeds
– Employs commercial video formats and
interchange standards
22

Summary
• Covered quite a variety of systems, both
hardware and software
• Managed to establish many noteworthy
performance metrics for partners to tout
• Illustrated the value and use of standards,
open systems, and agile development
methods
• Set the stage for Phase III and expanded
integration of the CTE infrastructure
23

Background

Continuous Transformation
Environment
Overview and Current
Direction
“A Cloud Computing Range”

• CTE Objectives
– Prototype development and experimentation: Innovation in a
collaborative environment by industry, government, and academia in
an open systems collaborative environment
– Integration, verification, test, and release of Commercial-Off-The-Shelf
(COTS) products for government use
– Rapidly prove operational utility of high technology solutions
– Open systems and standards compliance evaluation, documentation,
and capabilities matrix
• Solve GAO identified big integrator problem1
Organizing principle: “Give innovation a chance.”
• Consortium of large and small technology companies
and facilities partners
CTE at a Glance
261 Government Accounting Office-09-326SP, http://www.gao.gov/new.items/d09326sp.pdf

CTE Network
27
 Quality Technology Services (QTS) and Verizon (VZW) are the current facilities providers
 OCONUS Sites are VZW
 CONUS VZW: Engelwood, CO; Culpeper, VA, Miami, FL
 Remaining CONUS Sites are QTS, in particular the 1.3M sq. ft. Richmond, VA site

Main CTE Experimentation Lab Location
Richmond-High Density Multi-DataCenter Campus
500,000 Sq Ft of Planned Raised Floor
Multiple Distinct Data Center Buildings
(Current Basis of Design)
1
3
2 Office Space
1.3 Million Sq Ft Campus

Use Case Scenarios
1. Industry product for government use
2. Government product for industry use
3. Industry/government collaboration for government
use
4. Standards Evaluation for government and/or
standards body
5. Industry Compliance and Confidence testing
6. Government interoperability testing of industry
products to select replacements or build partnerships
7. Government operational evaluations/ttp
development of cloud solutions
8. Discovery and reuse through collaboration of
communities of interest

Use Case Scenario Development
• Find Opportunities
from Suppliers and
Users
• Develop and Evaluate
Opportunities
• Supply
Confidence/Knowled
ge to Consumer
• Field and Use

Government Opportunities
• Resource seed prototypes, enabling rapid industry driven
cloud solutions to DoD problems
• Establish DoD CRaDA with the CTE that enables mutual
benefit in:
– Open System, Standards, and Content
– Compliance testing results/methods
– Open system industry demonstrations for DoD consideration
– Open Government data/systems for industry consideration
• Establish Open Source/Open Systems Charter foundation
for DoD, including repository functions
31

Mutual Benefit
Industry/CTE Gain
• Direct Rewards to
Innovators
• Provide the Opportunity for
Agile Partnerships
• Access Government
Solutions/Information (e.g.
GOTS Software)
• Improve Organizational
Focus and Strategy
Execution
• Support Ongoing Innovation
Government Gain
• Implement Agile
Development and
Contracting Strategies
• Speed New Applications
and Systems from Concept
to Use
• Enhance Efficiency in
Fielding and Support
• Outpace Emerging Threats
• Optimize Resource
Employment

Way Forward/Next Steps
• Invite Government Leaders back to CTE for workshop –
November (OSD, Joint Staff)
• Develop Proposal for Prototypes – December (CTE)
• Develop Draft CRaDA – December (OSD, CTE)
• Develop White Paper on Open System/Open Source
Function – January (CTE, OSD)
• Seed into field activities (AF ACC, AF/Joint Training,
TRMC S&T) – May (CTE, OSD)
33

Summary
• Continue to evolve the capacity and capability of
the CTE with industry partners
• Form government partnership to create a
gateway for government-industry collaboration
• Establish a series of rapid prototypes for DoD and
non-DoD solutions
• Demonstrate open systems/open source
expertise
34

CTE Background
35

• QTS
• HP
• Hitachi Data Systems
• Verizon
• Oracle
• NetApp
• Solace
• Ciena
• IBM
• ISC8
• Quantum
• Spectralogic
• Blackridge
• EMC
• Nicera
• Juniper
36
CTE Infrastructure
Access to over:
 50,000+ Processing Cores
 100,000 reserve
 100+ Petabytes of Storage
 Over 30 Sites Worldwide
 Classified/Unclassified Facilities
And Growing…
Main facility in Richmond, VA:
 1.3M square feet
 110MW of power from 2 grid
elements
 Full onsite backup & UPS
 25,000 tons of chilled water
generation capacity per day

CTE Software Participants
• Cloud Infrastructure
– Amazon, Dell, EMC, HP, IBM, Oracle, Quantum, Red Hat, Aspera
• Data Storage and Manipulation
– Accumulo, Cloudera, Hortonworks, MapR, Sqrrl, Vertica/HP
• Document Manipulation and Analytics
– HyLighter, IxReveal, Digital Reasoning
• Video Management and Analytics
– Pixia, SIG, Aspera, Pixlogic, Flexanalytics, Terramajic, KSI Video
• Advanced Analytics Tools
– AgileX, HP, Attivio, IBM, Optensity, Oracle, Solace
• Security
– Blackridge, CyberPoint, IBM, ISC8, Red Hat, Security First, Verizon
37

Current Technical Emphasis
• Complete the CTE System Integration with High-
Speed Interfaces
– All the systems have 1G interfaces, will implement 10G by
the end of Q1 FY14
• Integrate all the systems into the central
provisioning service
• Implement the ability to provision virtual and
physical machines with complex applications and
databases pre-loaded
• Integrate machine provisioning with software
defined networking (SDN)
• Enable a vastly simplified complex
system/architecture implementation capability
38