Software & Hardware architecture & blueprint for an IoT/SCADA-based Management Platform for TowerCo's and Mobile Operators

1. TOWER OPERATIONS CENTRE (TOC) - ARCHITECTURE &
BLUEPRINT
Shahid Abbasi
2016

2. • Provide high degree of automation & optimization of business processes
• Support customization of business processes for individual departmental
needs
• Minimize management reaction times to network events
• Allow for geographical dispersion of control over aspects of network operation
• Share information across functional & organizational boundaries
• Provide isolation mechanisms to minimize security risks
• Provide mechanisms to locate and contain infrastructure faults
• Proactive service assistance and interaction with customers
• Allow for management of heterogeneous devices, sensors and services within
a passive infrastructure environment
• Allow for technological and functional changes
• Include capabilities to enhance early implementation and allow future
refinements (Day 0 & Final State Implementation)
• Allow for Opex & Capex optimization through use of hybrid energy
technologies and telemetry / remote monitoring (IoT) sensors, transducers &
actuators
Management System (TOC-MS) - Architecture Rqmnts

3. HIGH LEVEL SYSTEMS FUNCTIONALITY
Business Support Systems
• Asset & Inventory
Management
• Lease Management
• Document Management
• SLA Enforcement
• Billing & Settlements
• Financial Management (Capex
& Opex tracking)
• Workflow Management
• Site Planning
• GIS
• Reporting
• Management Dashboard
Operations Support Systems
• Site Design
• Site Build-out & Implementation
• Site Acceptance
• Network Infrastructure Surveillance,
Monitoring & Management
• Fault Management
• Trouble Ticketing & Tracking
• Energy Management
• Fuel Management
• Security & Access Management
• SLA Monitoring & Enforcement
• Vendor Management
TOC-MS High-Level System Functionality

4. CUSTOMER SERVICE CENTER
• End-to-end remote site‘ monitoring at a single central location - a solution that is a
combination of Remote Sensoring Unit & Central Application is required for the
implementation of a Tower Operations Center (TOC)
• A robust Business Management System (MS), which includes automation of modules
like Fault Management, Trouble Ticketing, Energy Management, Fuel Management,
Work Force Management, Supply Chain, Asset Management, GIS, etc
• Monitor all the technical parameters of the different equipment installed at the Cell
Sites
• IoT Instrumentation in the form of host of Sensors, Transducers, Actuators
• to collect parameters provided by these equipment and convert them into an
understandable and known format
• Monitoring and Control Equipment which can be hosted at each Cell Site
• to collect all the inputs in different formats from the sensors, transducers and
Actuators
• to forward these parameters to an upstream application for centralized
monitoring & management
• Connectivity Mechanism
• to transfer the collected information from the equipment placed at the Cell Site to
the Centralized Application installed in the TOC
TOC-MS Business Goals

5. CUSTOMER SERVICE CENTER
• A Centralized Application
• to process the information & events, collected from all the cell sites, generate
alarms, issue trouble tickets, send alerts to the workforce, issue corrective
actions back to the monitoring equipment, etc.
• A Cloud-based Infrastructure
• to seamlessly deliver TOC-MS features & functionality to specific locations,
irrespective of MIS Infrastructure’s physical deployment, while ensuring data
integrity, back-up & redundancy
• A battery of users and operators
• to understand the alerts and take corrective actions accordingly
• Would handle all Level 1 & 2 Passive, and Level 1 Active infrastructure alarms and
would deliver the MIS to monitor the performance and KPIs of the Infrastructure and
the Vendors involved in various activities of the operations.
• Backbone for TowerCo’s day-to-day Operations and would help to achieve the KPIs
and SLAs as committed to the customers by the TowerCo.
TOC Business Goals (Cont’d)

6. Tower Operations Centre – Initial State Day 0
SLA / KPI
Site Access
Fuel
Site Assets
Environment
Power
O&M Vendor’
Personnel Embedded
TowerCo Personnel
Embedded
Network Operations Centre
(Operator)
Help Desk
(TowerCo)
CRITICAL
MINOR
WARNING
NOTIFICATIONS
Electronic
Bonding
(Alarm & TT)
Infrastructure O&M Service
 Passive infrastructure surveillance
 First line maintenance
 Second line maintenance
Infrastructure Planning & Optimization
 Infrastructure capacity management
 Infrastructure expansion management
 Routine network optimization
Power / Energy & Fuel Optimization
 DG, fuel & power management
 Hybrid power systems
Spare Parts Management Service
 Spare parts dimensioning
 Warehousing
 Repair & Return
 Inventory management
Other Operational Services
 Instrument & tools management
 Documentation management
 Site access & security management
 SLA & KPI management
 Asset Management

7. Tower Operations Centre – Final State
Integrated real-time site monitoring and reporting
Telemetry
Controller
Tower Operations Centre
(Primary)
Tower Operations Centre
(Back-up)
 Telemetry transmits all sites’ O&M data to both TOC 1 & 2 for redundancy purposes.
 TOC manages, monitors, controls, measures and maintains various equipment on sites.
O&M Vendor’
Personnel
Embedded

8. ENERGY EFFICIENCY MANAGEMENT ASSET MANAGEMENT
ENVIRONMENTAL MANAGEMENT FUEL MANAGEMENT
TOC – 360o Operations Management
Support for complete business management of Tower Operations

9. HUMAN RESOURCE MANAGEMENT SMART SITE SECURITY
FINANCIAL MANAGEMENT OPERATIONS
TOC – 360o Operations Management (Cont’d)
Support for complete business management of Tower Operations

10. TOC – Key functions and features
Support for complete business management of Tower Operations
ENERGY EFFICIENCY
MANAGEMENT
 Generator and fuel management
 Rectifier alarms and variables
 Battery bank management
 Auto phase selector
 Hybrid management
 Grid availability & power meter’ reading
verification
SITE MANAGEMENT
 Detailed display – site assets
 QR barcode – individual site
elements
 Site topology configuration
 Segregated maintenance
procedures
 Real-time & historical trends
ENVIRONMENTAL MANAGEMENT
 Free air-cooling utilization
 Room thermal mapping
 Remote configurations & settings
 Fire, smoke, and flood detection
 Cooling units consumption
 Temperature & HUMIDITY monitoring –
different locations
FUEL MANAGEMENT
 Fuel quality
 HYBRID remote on/off ability
 MODBUS compatibility
 Fuel consumption, fuel drop, &
re-fuel measurement
 Versatile tank shape support

11. TOC – Key functions and features
Support for complete business management of Tower Operations
RESOURCE MANAGEMENT
Personnel tracking
Contractor working hour’ monitoring
Alert and trouble ticket assignment
Maintenance activity summaries per
technician
Real time access to human resources
SITE ACCESS & SECURITY
Tamper proof systems
Site protection mechanisms
Recorded back up systems
Site access protocols
Physical / guarded sites
OPEX MANAGEMENT
Opex reduction analysis
Site behavior analysis
Alternative resource site management
Expense analysis & forecast
SLA & KPI goal analysis
Resource efficiency optimization
Multi-tenant management
OPERATIONS
Trouble ticket management
O&M supplier management
Alarm management
Auto / remote alarm control
Business models:
monthly/maintenance
Site-process control
Business intelligence
Financial procurement

12. TowerCo Functional Requirements (Final State)
Operations and
Maintenance
- Monitoring
- Alerts Mgmt
- Alarm Mgmt
- Trouble Ticket
- Fuel Mgmt
- Energy Mgmt.
- Env Mgmt
- Site access
and Security
- Dispatch
- Workforce
Mgmt.
- Help Desk
- SLA Mgmt.
- Site Mgmt
Inventory and
Asset
Management
- Physical
Inventory
- Logical
Inventory
- Service Inventory
- GIS visual
- Failure
inventory
- Sites plan
- Hierarchical
view
Site Financials
Management
- Site Capex &
Opex
- NOC & Permits
- Procurement
- Rent Mgmt.
- Expense
Analysis and
Forecast
- Site Lease
Mgmt.
Enterprise
Resource
Planning
- Company
Financials
- HR
- Procurement
- Document
Mgmt
Performance
Management and
Integration
- KPI reporting
- Business
Intelligence
- Financial
planning
- Modelling
- Integration with
O&M, I&AM,
SFM, BM
security
integration
monitoring and system management
infrastructure

13. TowerCo Functional Requirements (Day 0)
Operations and
Maintenance
- Monitoring
- Alerts Mgmt
- Alarm Mgmt
- Trouble Ticket
- Fuel Mgmt
- Energy Mgmt.
- Env Mgmt
- Site access
and Security
- Dispatch
- Workforce
Mgmt.
- Help Desk
- SLA Mgmt.
- Site Mgmt
Inventory and
Asset
Management
- Physical
Inventory
- Logical
Inventory
- Service Inventory
- GIS Visual
- Failure
inventory
- Sites plan
- Hierarchical
view
Site Financials
Management
- Site Capex &
Opex
- NOC & Permits
- Procurement
- Rent Mgmt.
- Expense
Analysis and
Forecast
- Site Lease
Mgmt.
Enterprise
Resource
Planning
- Company
Financials
- HR
- Procurement
- Document
Mgmt
Performance
Management and
Integration
- KPI reporting
- Business
Intelligence
- Financial
Planning
- Modelling
- Integration with
O&M, I&AM,
SFM, BM
security
integration
monitoring and system management
infrastructure
Capabilities required on Day 0

14. TOC-MS Actor & User Context
TOC-MS
Telemetry /
IoT
Sensors
Field
Engineers
Business
users
Inventory /
Asset users
Help desk
& TOC
users
Mgmt users
Management
reporting
dashboard
Financial reporting,
site opex, supply
chain, O&M
vendors, payroll
Inventory,
asset mgmt
OPCO
Trouble ticketing,
incident/change
mgmt, alerts,
events, alarms
SLA reports
Call dispatch,
call status
update
Environment,
fuel, security,
energy data,
video stream
Actors (Users and instruments) interacting with TOC-MS

15. TOC-MS IT’ Technology Protocol Context
TOC-MS
Telemetry /
IoT
Sensors
Field
Engineers
Business
users
Inventory /
Asset users
Help desk
& TOC
users
Mgmt usershttps, Ethernet,
3G/LTE
OPCO
VPN over
Internet, http/
web services/
file transfer
https, 3G/LTE,
SMS, SMTP,
Telephone
MQTT, SNMP,
RTP, GPRS,
3G/LTE
https, Ethernet,
3G/LTE
https, Ethernet,
3G/LTE
https, Ethernet,
3G/LTE,
Telephone
Technology Protocols interacting with TOC-MS

16. TOC-MS Architecture Overview
Humidity
Sensor
Tower Site
Field Office
TowerCo Operations Center
OPCO
Fuel Level
Sensor
Fuel flow
Meter
SCM/
gateway
Temp
Sensor
Smoke/Fire
Detector
Utilities Co.
Smart meter
Battery power
Sensor
Door Access
Switch
Security
cameras + IVA
3G/LTE
IVA + Video
Recorder
User -
Browser + Mbl
Head End
DISCO
DG Controller
Alert Mgmt.
System
Help Desk
System
Fin. Mgmt.
System
RFID
Controller/
QR Scanner
Inventory/
Asset Mgmt
User –
Browser + Mbl
User –
Browser + Mbl
ODS
Reporting
System
Reporting
Data Store
User access -
Web Browser
MDMS Billing
User –
Browser + Mbl
User Access -
Web Browser
+ Mobile
Identity
Server
Backup
Server
Security
Systems management
Messaging
Server
Messaging
Client
Systems and
NW monitoring +
mgmt
Proactivebillretrieval
TowerCo A/C
meter
Multi-channel
DC Meter
Rectifier

17. TOC-MS High Availability and DR
Redundant IP links
IP based data replication
Devices/users/
stakeholders
External Systems External Data Sources
Normal operation
DR operation
Data replication
• No SPOF configuration within DC
• Disaster Recovery Site as per Industry Best Practices
TowerCo Operations Centre
IVA + Video
Recorder
User -
Browser + Mbl
Alert Mgmt.
System
Help Desk
System
Fin. Mgmt.
System
Inventory/
Asset Mgmt
User –
Browser + Mbl
User –
Browser + Mbl
ODS
Reporting
System
Reporting
Data Store
User –
Browser + Mbl
Identity
Server
Backup
Server
Security
Systems management
Messaging
Server
Systems and
NW monitoring +
Mgmt
TowerCo Operations Centre
IVA + Video
Recorder
User -
Browser + Mbl
Alert Mgmt.
system
Help Desk
System
Fin. Mgmt.
system
Inventory/
Asset Mgmt
User –
Browser + Mbl
User –
Browser + Mbl
ODS
Reporting
System
Reporting
Data Store
User –
Browser + Mbl
Identity
Server
Backup
Server
Security
Systems management
Messaging
Server
Systems and
NW Monitoring +
Mgmt

18. The TOC arrangement will consist of:
• Main video wall display
• Operators sitting (according to technology, geography or any
subject such as redundancy)
• Shift manager post (in the middle and elevated)
• NOC manager room (tied to the main hall)
• Back office support (second level experts)
• Meeting room/visitors room (tied to the main hall)
• Administrator room
• Server room
• Air conditioner / Climate control
• Audio and noise planning
TOC Set-up

19. INFRASTRUCTURE COMPONENTS - OPTIONAL
• Acoustic Ceiling construction
• Floating Floor (tiles, carpets)
• Plaster Walls
• Doors with different acoustic isolation
• Windows between the different areas
• Furnishings
• Announcement System
• Control Systems
• Fire Detection System
• Low voltage cabling systems
Physical TOC Arrangement

20. MULTIMEDIA SYSTEMS - OPTIONAL
• Audio Systems
• Intercom System
• Audio Broadcast System
• Audio Recording Systems
• Video Systems
• Video Conference
• Video Recording
• TV Sets, Projectors, Screens
• Smart Board
• Plasma Panel / LCD
• TV / Satellite Connection
• Communication components
• WAN/LAN Network
• Servers, Computers, Printers
TOC Infrastructure Components

21. Deployment Options
Prod and DR on premises (Private
Cloud):
TowerCo Owned and Operated private cloud:
- Heavy CAPEX Up front
- Assets on TowerCo’s books
- Higher time to market due to Data Centre build
As a Service:
- OPEX based option
- Assets can be on TowerCo’s books or service provider
- Still higher time to market due to DC build
Prod on premise, DR on Cloud:
- Relatively lower CAPEX
- DR environment can be minimal until disaster happens
saving on:
- HW infrastructure cost
- Licensing costs
- Seamless movement of workloads between private and
public cloud possible
- Gives flexibility to burst into cloud for incidental workloads
- Local vs overseas cloud providers can be evaluated
Prod and DR on Cloud:
- Lowest CAPEX option, possibly no assets on TowerCo’s
books
- As a service, SLA based operations
- Endless scalability for additional workloads
- No need to separate DR environment, SLA covers that
- Heavy dependency on network for mission critical
applications
- If remote cloud, network latency could possibly impact
response time, specially for telemetry based events
requiring immediate action
Prod on Cloud, DR on Premise:
- Unusual option but has some merits for consideration
- In case of network outage, failover to local site
- Local resources can be re-purposed for trivial workloads
unless there is a failover

22. Recommended Approach
Prod and DR on premises (Private
Cloud):
TowerCo Owned and Operated private cloud:
- Heavy CAPEX Up front
- Assets on TowerCo’s books
- Higher time to market due to Data Centre build
As a Service:
- OPEX based option
- Assets can be on TowerCo’s books or service provider
- Still higher time to market due to DC build
Prod on premise, DR on Cloud:
- Relatively lower CAPEX
- DR environment can be minimal until disaster happens
saving on:
- HW infrastructure cost
- Licensing costs
- Seamless movement of workloads between private and
public cloud possible
- Gives flexibility to burst into cloud for incidental workloads
- Local vs overseas cloud providers can be evaluated
Prod and DR on Cloud:
- Lowest CAPEX option, possibly no assets on TowerCo’s
books
- As a service, SLA based operations
- Endless scalability for additional workloads
- No need to separate DR environment, SLA covers that
- Heavy dependency on network for mission critical
applications
- If remote cloud, network latency could possibly impact
response time, specially for telemetry based events
requiring immediate action
Prod on Cloud, DR on Premise:
- Unusual option but has some merits for consideration
- In case of network outage, failover to local site
- Local resources can be re-purposed for trivial workloads
unless there is a failover
• Start with a local cloud
service provider
• Build a site facility in parallel
and migrate to owned facility
when ready
• Engage a systems integrator
to build, manage and support
the IT environment for the
first 3 years
• Build skills and capability in
parallel
• Take over management of IT
operations

23. ID Fun-
ction
Description Metric Possible Candidate Products
1 Infra Directory Server (Active Director or LDAP) # of Users Microsoft/ Open source
2 Infra Backup and Archiving Server TB Data Veeam VBR / IBM TSM / HP /
EMC Networker
3 Infra Messaging and collaboration Server (MS Exchange + Lync or
Lotus Domino + ST)
# of Users Microsoft Exchange+Lync/ IBM
Lotus + Sametime
4 Appl Service Desk System # of Users HP Service Desk / BMC Remedy/
IBM Control Desk / Tarantula/ AIO
5 Appl Inventory, Asset and Maintenance Management System # of Users IBM / Cisco / Accruent / AIO
Systems/ Tarantula
6 Appl Financial Management System Users / # of cores SAP/ MS Dynamics/ Oracle ERP
7 Appl Reporting Engine # of Users IBM Cognos/ SAS/
SAP Business Objects/ Watson
Analytics
8 Appl Operations Control Center / Alert Management System # of users IBM IOC (websphere + Netcool)/
AIO Systems/ LCC/ Tarantula
9 Infra Server Virtualization Software # of proc sockes Vmware vSphere/ MS Hyper-v/
KVM
10 Infra Operating System (dependent on Application compatibility) # of VMs MS Windows / Linux
11 Infra Applications Systems and Network Management Tool # of servers+ Nw
devices
IBM Tivoli/ HP Netview/ EMC
Smarts/ Solarwinds
Products BoM (Software)

24. ID Fun-
ction
Description Metric Possible Candidate Products
12 MW Database Product # of cores MS SQL/ MySQL (OS)/ Oracle/
IBM DB2/ Informix
13 Infra Security certificates Thawte/ Verisign
14 Infra Video Management System # of sites/
cameras
ACTi/ Avigilon
15 MW Integration Software # of cores IBM Integration Bus/ Tibco BW/
Oracle Fusion
16 MW Document and Workflow management # of users MS Sharepoint, IBM Filenet, EMC
Documentum
Products BoM (Software Cont’d)

25. ID Fun-
ction
Description Metric Possible Candidate Products
1 Infra Firewall cluster for Network security with virtualization Concurrent
sessions
Juniper / Cisco
2 Infra Load balancer cluster for Web based traffic with virtualization Concurrent
sessions
Brocade / Cisco
3 Infra Edge Routers Concurrent
sessions
Juniper / Cisco
4 Infra Network Fabric Switches (10 Gbps Ethernet) for LAN # of ports Juniper / Cisco
5 Infra Network Fabric Switches (10 Gbps FC over Ethernet) for SAN # of ports Juniper / Cisco
6 Infra Compute Servers (RISC or x86) for all functions # of proc
cores
X86 (HP, Lenovo, Dell)/ IBM
Power
7 Infra SAN attached storage TB Data IBM v7000/ EMC VNX/ Dell
8 Infra Network connectivity (WAN) # of links,
Data thruput
Connectivity vendor
9 Infra Scanners # of units HP/ Canon
10 Infra Printers # of units HP/ Canon
Products BoM (IT Hardware)

26. ID Fun-
ction
Description Metric Possible Candidate Products
1 Infra Sensor-cum-communication (SCM) module # of sites Tarantula/ AIO/ Commscope/6th
Energy/LCC
2 Infra Fuel-level sensor # of fuel tanks Various
3 Infra Fuel flow meter # of fuel tanks Various
4 Infra Temperature sensor # of sites Various
5 Infra Environmental sensors # of sites Various
6 Infra Smoke sensor # of sites Various
7 Infra Door switch & Magnetic Locks / Access control # of sites Various
8 Infra Dual AC Energy meter – Mains & DG # of tenants Various
9 Infra 4-Channel DC Energy meter # of tenants Various
10 Infra Hybrid Energy Controllers (functionality may be available in SCM) # of sites Emerson/Eltek/Huawei/AIO/
Tarantula
11 Infra Security Cameras # of sites Indigovision/ RIVA/ Avigilon/ ACTi
Products BoM (Telemetry, Sensors & Actuators)